Index Load configuration files for indexing
Index Load configuration file | Data Load definition file |
---|---|
Environment configuration file (wc-indexload-env.xml) | wc-dataload.xsd |
Profile configuration file (wc-indexload-profileName.xml) | wc-dataload-env.xsd |
Profile item configuration file (wc-indexload-businessobject.xml) | wc-dataload-businessobject.xsd |
Environment configuration file (wc-indexload-env.xml)
The wc-indexload-env.xml file contains environment control information and global properties required by Index Load, including a common data writer and data source to be used to persist the data.
The wc-indexload-env.xml file does not typically require customization, you can use the default sample file as-is.
Profile configuration file (wc-indexload-profileName.xml)
The wc-indexload-profileName.xml file contains configurable performance attributes and load item configurations.
Profile names that you define in configuration files are then substituted in as a URL parameter when calling Index Load in a web browser.
The load item configurations are listed under the load order section of this file. They are processed in the same order as they are specified.
It can contain one or multiple LoadItem definitions, with every LoadItem configuration specifying the specific loaditem configuration and coreName target. Multiple LoadItems are run in parallel, without sequence.
- batchSize
- The threshold when documents are soft committed in memory.
- commitCount
- The threshold when documents are hard committed to disk from memory.
- ThreadLaunchTimeDelay
- The amount of time in milliseconds to wait before launching another new thread, to avoid overloading the system at startup.
- OptimizeAfterIndexing
- Indicates whether Index Load performs index optimization after
commit.Note: Performing optimization after a full indexing improves runtime performance; however, it increases the overall indexing time.
- StatusRefreshInterval
- The maximum amount of time in seconds to wait before refreshing the current Index Load status and display it in the administrative log.
- IndexHeightCacheHint
- A number that hints the system to determine the size of the applicable caches for index height used during indexing.
- IndexWidthCacheHint
- A number that hints the system to determine the size of the applicable caches for index width used during indexing.
Profile item configuration file (wc-indexload-businessobject.xml)
- ParallelThreads
- Reads data in parallel. It specifies the maximum loader thread number which can be dispatched by the search work manager. The loader thread will read data in parallel, sharing the same data writer.
- ParallelLowerRangeSQL
- SQL queries that get the first keys.
- ParallelUpperRangeSQL
- SQL queries that get the end keys.
- ParallelNextRangeSQL
- An SQL statement that determines the next available identifier when an empty range ID is detected from the parallel range. Typically, the nextStartKey value is the firstKey, and the nextEndKey is the firstKey+prefetchSize-1.
- ParallelLowerRange
- A hardcoded value that keeps track of the lower range keys. If defined, it is an absolute number for the lower range and overrides the value of ParallelLowerRangeSQL.
- ParallelUpperRange
- A hardcoded value that keeps track of the upper range keys. If defined, it is an absolute number for the upper range and overrides the value of ParallelUpperRangeSQL.
- ParallelPrefetchSize
- Determines how much data to read in one run, when the reader performs a query from the database. If defined, the runtime will break up the entire data range into fragments to avoid overloading the database sort heap with too large a query result set
- ParallelDeltaUpdate
- Determines whether the SQL result set will be merged into an existing indexed document that contains a matching primary key.
Sample configuration files
You can use the following sample configuration files for reference: IndexLoadSampleCode.zip.
The sample includes configuration files used by Index Load, and manual updates performed in the Indexing contract prices using Index Load task, for reference.