Multi-instance Pipeline Processes

The pipelined process runs multiple CPU-intensive operations, such as privacy blocking activities, which can cause performance bottlenecks in single-threaded configurations.

You can create additional instances of the pipelined process to distribute the processing load for all DNCA instances across available CPU resources.

For example, suppose that a single DNCA instance is generating 500 pageviews/second and is configured for intensive pipeline privacy processing, which is limiting its throughput to 200 page views per second. Adding two more pipeline instances (for a total of three pipelines) enables the handling of the overall page-view throughput.

One or more reassd processes (multi-DNCA instances) can feed its resulting HTTP hits to a single, shared memory (SHM) queue, which manages distribution to the available instances of the pipelined processes in round-robin fashion.

In an example of multi-DNCA instances, suppose that you have created four DNCA instances, which are generating 1000 pageviews/second. If a single pipeline can process 400 page views per second in your environment, two more pipelines can be added to manage processing the entire volume.

DNCA master and slave failover also supports multi-instance pipelines. See DNCA Web Console - Failover Tab.