Skip to content

Replay Rules

To deliver the highest fidelity replay without impacting the origin server or servers of the web application, replay rules can be used.

A replay rule is a modification to the visitor session data during replay, the goal is to achieve a replay of the website as close as possible to the original experience. For example:

  • The captured session data can contain references to a server that is not available from where HCL Discover users are viewing the session. To address this a replay rule is required to modify URLs pointing to the origin server to point to a different server.
  • If the session is from a site or application that requires authentication, an unauthenticated Discover user can generate a significant number of 401 - Access Denied browser errors during replay. Using a replay rule, you can instruct the replay to ignore or remove these pages.
  • The session can reference an external JavaScript file, which you may or may not want to reference during replay. Configuring replay rules can be used to suppress the JavaScript file or to modify it so that it works effectively during session replay.

There are many more uses for replay rules. In the sections below, you can review the replay rules that can be configured through the Replay Server to be applied to all replay sessions.

How rules apply

Replay rules do not change the stored session data. Rules are applied only during replay by the Replay Server to the data that interpreted for display to the Discover user.

The quality of session replay is dependent upon the nature of the site / application. Websites that employ sophisticated display technologies or rely on client user interaction events can require significant customisation of the common replay profile, replay rules and JavaScript UIC as their combined focuses are replay fidelity and data capture.

Creating rules

Replay rules can be configured in the following ways with Discover:

  • Replay UI (Advanced only) - When you have a session that is loaded into replay, you can use the context menus to create replay rules that are based on specific, highlighted data in the currently loaded session.
  • Configuration File - For Discover administrative users who are comfortable working with XML, replay rules can be configured in the native XML format by accessing the configuration file on the master Replay Server.

Editing rules

Discover does not recommend configuring replay rules through the XML file, as no data validation is completed on the edited file. Where possible, use the user interface method for configuring replay rules.

Applying rules

Using the Request, Response or Replay view you can configure replay rules through any of the context menus. Selecting an item using a right-click, and choosing one of the following options:

Option Description
Remove page from Replay based on request field Based on a field included in the request, you can selectively remove the page from display during replay. Note: This selection is available only if text is selected in Request view.
Test Response Modify Rules Through this dialog, you can review and test any response modify rules that are applicable to the current page. Note: This selection is available in Response view only.
UI Event Custom Highlight When a UI event is selected in the Navigation List, you can add or modify custom highlighting rules to be applied to the user interface event. Note: This selection is available through the context menu when you select a UI event in the Navigation List.
Host/Port Remapping From the Load Details screen, you can configure host/port remapping rules.

Removing pages

Through replay rules you can choose to remove one or more pages that are based on the URL pattern, request variable parameters, or both. By removing pages that are based on request variables, you can significantly remove the replay experience.

Data availability

Pages that are removed from replay are not removed from the session data. You can still search for and report on these pages.

If your site responds to multiple HTTP_HOST values, you must manually edit the profile to ensure that the ignore rule is correctly recorded against all server_name values.

For example, to remove from replay pages that are permanently moved or are redirect pages, which are determined based on specific values for the StatusCode request variable. This selection is available only if text is selected in Request view.

StatusCode=301 indicates a page is moved permanently. In the example, when the URL matches /store/index.php/electronics/cellphones/ and the StatusCode request variable in the [env] section is set to 301, then the page is removed from replay.

In these fields, you must specify the request variable section and variable name on which to match pages for removal from replay replay.

  • The request variable section must be specified.
  • Wildcards are not permitted.

For additional specification, you can enter a specific value for the request variable on which to match. In the previous example, a request variable value must be included, or all hits that contain StatusCode entries are dropped from replay, which would remove all valid hits that are returned from the web server.

  • Specifying this parameter is optional. If it is not specified, all values are matched.
  • When you specify values, you can use the standard wildcards: ? to replace a single character, and * to replace multiple characters.

You can use this field to specify whether the replay rule is applied to a specific URL or a pattern of URLs. If this value is not specified, the replay rule is applied across the entire domain.

  • Specifying this parameter is optional. If it is not specified, all values are matched.
  • When you specify values, you can use the standard wildcards: ? to replace a single character, and * to replace multiple characters.

Testing Response Modify rules

Through the Replay UI, Response Modify rules are applied to the current page can be tested by using the current hit as test data. This selection is available in only using the Response view, any Response Modification rules are created through the Portal Management page.

Using the Test Response Modify Rules dialogue, you can review the number of rules that are applied to current URL and complete and review tests results by using those rules. To test applicable Response Modify rules click Test in the Response Modify Rules dialogue, .

Custom UI event highlighting

For the selected UI event, you can add or modify custom highlighting rules. These rules can be used when the selected element requires special handling, such as calling a JavaScript function for a custom control. Custom highlighting rules can also be used for debugging or alerting purposes when specific user actions or values are displayed in a session.

The element ID is pre-populated with the unique identifier for the selected event through element ID or XPath. You can modify the Element ID field to use a regular expression, if needed, for matching multiple elements.

In the dialogue, you can specify the function that is evaluated by replay for purposes of highlighting this element. During evaluation, no setting of values, highlighting or clicking is performed. Discover provides a set of references that you can use in your JavaScript function.

Discover supports the following types of replay rules in the table below. You can determine whether the rule is supported for configuration through the Replay Server or replay.

Replay rule types

Type Description
Popup Page PopupURL PopupURL marks pages as popup, so they are not used for highlighting. This rule is available in the replay Rule Profile Editor.
Remove this page from Replay   You can create a regular expression to match URLs that you do not wish to display in the Navigable Pages List or during replay.
HighlightonlyURL HighlightOnlyUrl For AJAX-based pages, you can configure a page to apply highlighting only to the content retained from the previous page. In this manner, you can simulate the application and the capture of UI events. This rule is available in the replay Rule Profile Editor.
Ignore ReqVar for URL Match IgnoreReqVarForURLMatch During replay, the replay client attempts to match requests to responses in the session data. In some cases, you can improve matching by providing a URL pattern, which is used to remove request variables, such as timestamp information, from the set that is used to match the request to the response. This rule is available in the replay Rule Profile Editor.
ResponseMod ResponseModify ResponseModify rules are used to complete special modifications to the response to enhance the replay that is displayed to the Discover user. For example, you can configure a ResponseModify rule to disable JavaScript used to break their pages out of frames. This rule is available in the replay Rule Profile Editor.
RequestMod RequestModify The Request Modify rule is useful for fixing anomalies in data or fixing custom content not handled by Replay. For example, you can use the Request Modify rule to modify the URL of a page or make corrections to incorrect data in the JSON UISDK. The Replay server can then load the modified data and perform its task. This rule is available in the replay Rule Profile Editor. Here are the parameters defined: URL This is a regular expression defining which URLs this rule is to be applied to. Reqvar(optional) The name portion of a name/value pair in the request to evaluate. Value A regular expressions. If the expression matches the value for reqvar the rule is applied. Pattern A regular expression used to match the text in the request to be replaced. Replacement This is a literal string that will replace the text matched by the pattern. Meta-characters (such as \r, \n, \t) are converted to the corresponding characters prior to replacement. Occurrences Replace either the first occurrence or all occurrences.
External File Modification ExternFileModify ExternFileModify rules can be configured to complete additional processing to external files that can interfere with replay. This rule is available in the replay Rule Profile Editor.
Dynamic Response Modification Rules DynResponseMod When replaying a session that includes data that is delivered from a third party, the contents from the third party must be associated with a specific request. If the third-party content is required to accurately represent the customer's experience, a dynamic response modification rule must be configured to acquire the content at replay time.
Dynamic External File Modification Rules DynExternalFileMod If the main page loaded into ... references a separate file that is not part of the session, the file may require modification before it is loaded into .... For example, if a JavaScript file referenced by a page in a session contains a domain reference, that reference can result in a JavaScript error during replay. You can design dynamic external file modification rules to apply to the referenced JavaScript file that modify the file before it is loaded into ....
Remove this page from Replay IgnoreURL IgnoreURL removes pages that should not be replayable from the Viewable Pages list. This rule is available in the replay Rule Profile Editor.
Host Remap RemapHost RemapHost remaps the host that is named in the HostProfile node to some other host. For example, you can use this mapping to replay session data from a backup server, instead of the origin server, which can impact usage metrics. This rule is available in the replay Rule Profile Editor.
Portal Remap RemapPort RemapPort remaps the port number of the host that is named in the HostProfile node to some other port number. This rule is available in the replay Rule Profile Editor.
URL Remapping RemapURL You can use the RemapURL rule to redirect static content retrieval. This is needed when the external content is not available or accessible from the live web site, and a copy is saved on another server where it can be reached.
Host Protocol Protocol Protocol forces the protocol to a specific value: auto, http, https. This rule is available in the replay Rule Profile Editor.
Frame Rule FrameRule FrameRule is used to force a URL to alway load into particular frame
BlockedURL BlockRemoteURL If needed, you can specify a URL or set of URLs that are prevented from reaching out to the origin server. If the replay client is unable to locate the response in the session data, no effort is made to reach the origin server for the missing content. This rule is available in the replay Rule Profile Editor.
New Host HostProfile Add a host to monitor. The name of the host must be the full domain name of it (for example, www.example.com. Each host has its own set of profile rules, so you must add or copy rules from any existing host to the new one. This rule is available in the replay Rule Profile Editor.
Whitelist Rule whitelistName By default, the Replay Server operates in Blacklist mode, in which all URLs are permitted to contact the origin server. In many environments, this ability to touch the origin server during replay is not desirable or even permitted. In Whitelist mode, you can configure URL patterns that are permitted to contact the origin server, and all other URLs are blocked. This rule is available in the replay Rule Profile Editor.
Custom UI Event Highlighting UIElementCustomHighlight Replay Rule for UI Events: For the selected UI event, you can add or modify custom highlighting rules. These rules can be used when the element being highlighted needs special handling, such as calling a javascript function for some custom control. They can also be used for debugging or alerting purposes when specific user actions or values is displayed in a session. This rule is available in the replay Rule Profile Editor.
Ignoring UI Elements during Replay Replay Rule for UI Events Replay Rule for UI Events: You can choose to ignore selected UI elements during session replay. When this option is selected, the UI Event Element Ignore dialog is displayed with the regular expression pattern to identify the UI element.
Keystroke Breakouts Replay Rule for UI Events Replay Rule for UI Events: UI Capture supports the capture of Intellisense keystrokes as UI events. Keystrokes applied with UI elements, such as textboxes and form fields, are bundled together into a single UI event for capture. To support appropriate replay of the visitor experience, ... can be configured to break out these aggregated keystroke events into individual UI events for each keystroke. When keystrokes are broken out in ..., you can see the characters that are displayed in the order that the visitor entered them.
Remap URL   You can use the remapping URL feature to remap the URL of content external to the captured pages of a session to a new destination. Remapping URLs is commonly used in situations where the external content is not available or accessible from the original site, and a copy is made on another server to which you can remap the URL.
PatchResponseWithPostValue PatchResponseWith PostValue This rule allows you to set up patterns in the request, from which values are grabbed. These values are patched in the response. This work is handled by the plug-ins, since the type of the data can vary, and each plug-in best knows its data type. This rule has several attributes. requestValue - Use this attribute for query string values, or values where the plug-in parses and 'flattens' the request into name-value pairs. This is a simple string match, not a regex. Use this attribute or requestPattern, not both. requestPattern - Use this attribute for request body values. This is a regex pattern, which is used on the raw request body as a whole. Use this attribute or requestValue, not both. When responsePattern is empty, this pattern is used as the response replacement pattern. responsePattern (optional) - Use this attribute when the response replacement pattern needs to differ from requestPattern. You can also use this rule when a value in a query string needs to be patched into a jsonp response. The responsePattern attribute is used to specify what to replace in the response. This rule is available in the replay Rule Profile Editor.
MobileMod Rule MobileMod MobileMod Rule corrects the dimensions of mobile devices that are based on user agent information during Replay. This rule is available in the replay Rule Profile Editor.

The Replay Server supports the creation of a custom set of replay rules to apply to replay sessions. These rules are managed by the master Replay Server, which publishes them on request to the other slave Replay Servers.

Multi-replay servers

If you are deploying white lists in an environment with multiple Replay Servers, additional configuration is required.

The URL can include the wildcard character (*) to represent a pattern of URLs, such as home*.

The host name is used instead of the host pulled from the domain of the session. Since the host is used for the \<base> to indicate the source of the content, this value can be used to remap the content in the session to a site other than the original site For example, if the original site is obscured by proxies, this setting can be used to remap to a static, available resource. It can also be used to map to an available server within the proxy from which images and other content can be loaded.

  1. You can modify content in the response by using simple search-and-replace.

Special regular expression characters such as ( or ) must be escaped by using a backslash. For example:

Escaped example
   \(

Adding a white list

You can add a white list domain rule to the replay server profile.

To add a white list domain rule to the Replay Server profile:

  1. Add a new XML node named DomainWhitelistURLPattern within the ReplayServerProfile node. Do not add the new XML node inside a HostProfile.
  2. Apply the following syntax rules
Escaped example
   \<DomainWhitelistURLPattern urlpattern=".*\.foo\.com"/>

urlpattern is a regular expression used to match allowed domain names.

More than one DomainWhitelistURLPattern rule may be defined if needed.

To enable the whitelist rules, set WhiteListEnable to 1 in the Replay Server global options.

Whitelist rules

By default, the Replay Server operates in a mode in which all URLs are permitted to contact the origin server (Blacklist). In many environments, this ability to touch the origin server during replay may not be desirable or even permitted.

In Whitelist mode, configured URL patterns are permitted to contact the origin server, and all other URLs are blocked.

Whitelist mode

Whitelist mode must be enabled in each Replay Server within your environment, whilst most implementations will only have one (1) larger deployments can have several.

When you enable Whitelist mode for the Replay Server, you can specify the set of URLs that are permitted to contact the origin server. By using regular expressions, you can specify a set of URL patterns that are permitted to query the origin server during replay. For the listed URL or URLs, the Replay Server references the data that is stored on the origin server during replay.

The URLs that do not match the whitelist are effectively blacklisted, which can be used to protect the source application from triggering web analytics metrics and Discover data counts.

RegEx testing

As noted elsewhere, poorly specified regular expressions can significantly affect server performance. Thorough testing should be carried out and where possible expressions should be simplified.

  • Special regular expression characters such as ( or ) must be escaped by using a backslash.

    Escaped example
       \(

  • If the pattern value is empty, the default pattern is used. This pattern causes the Replay Server to contact the origin server for image and static content, which is not stored as part of the session data.

    Escaped example
       \.

Rule Examples

RemapURL

  1. Setup a new folder in /Program Files (x86)/HCL Unica Discover/Portal/web on the replay server to hold your assets (CSS files etc ..)
  2. Add a RemapURL rule in Advanced Replay > Replay Rules.
  3. Just as an FYI the replace string in this example is: https://example.domain.com/Portal/Aurora_Assets/extra-local.css

Using XML

Best practice is to use the UI to add any rules, not the XML snippets as show below for further explanation. Cut & paste can cause errors.

RAWXML
<HostProfile name="example.domain.com" id="10">
   <RemapURL id="11" urlPattern="./css/." remapFormat="https://example.domain.com/Portal/Aurora_Assets%7"/>
</HostProfile>
<HostProfile name="example.domain.com" id="12">
   <RemapURL urlPattern=".*head_css" remapFormat="https://example.domain.com/Portal/Aurora_Assets/extra-local.css" id="13"/>
</HostProfile>