Data Aggregation and Retention

The Discover reporting system is driven by data collected from the Discover Processing Servers. Collected data consists of statistical information that is generated while sessions are processed. The data is generated in one-minute buckets and extracted into the reporting database every five minutes.

  • The actual text of the sessions remains on the Processing Server and is not migrated to the reporting databases.

The collected data is aggregated into two types of reporting data:

  • hourly
  • daily

After data has aged a pre-defined period of time, the data is removed from the database so that it can be kept to a manageable size.

The length of the retention period is positively correlated to the size of the database; retaining more data can result in a very large database, particularly in the tables that store hourly data.

This section provides some guidelines in configuring data retention and aggregation.