Configuration Elasticsearch et NiFi minimale

La configuration minimale pour Elasticsearch et NiFi est fournie par défaut dans votre génération initiale. Elle utilise la configuration suivante.

Tailles de segment

NIFI Heap – 9GB 
Elasticsearch Heap   - 12GB

Modifications d'unité d'exécution des groupes de processeurs

Flux NiFi > Service de journalisation

Nifi Flow>>Logging Service>>Generate Log Message: Configuration/Concurrent Tasks = 4 
Nifi Flow>>Logging Service>>Batch Bulk Document – Log: Configuration/Concurrent Tasks = 4

Tous les autres processeurs sont définis sur Tâches simultanées= 1

Flux NiFi > Service en masse – Produit

Nifi Flow>>Bulk Service – Product>>Bulk Elastic Search>Track Bulk Request: Configuration/Concurrent Tasks = 4
Nifi Flow>>Bulk Service – Product>>Bulk Elastic Search>Post Bulk Elasticsearch-Product: Configuration/Concurrent Tasks = 16`
Nifi Flow>>Bulk Service – Product>>Bulk Elastic Search>Set stage.end.time : Configuration/Concurrent Tasks = 4
Nifi Flow>>Bulk Service – Product>>Bulk Elastic Search>Analyze Bulk Response: Configuration/Concurrent Tasks = 4
Nifi Flow>>Bulk Service – Product>>Bulk Elastic Search> Post Refresh Elasticsearch: Configuration/Concurrent Tasks = 4
Nifi Flow>>Bulk Service – Product>>Bulk Elastic Search>Retry Flow File: Configuration/Concurrent Tasks = 4
Nifi Flow>>Bulk Service – Product>>Bulk Elastic Search>Retry Flow File: Configuration/Concurrent Tasks = 4
Nifi Flow>>Bulk Service – Product>>Bulk Elastic Search>Route On Environment: Configuration/Concurrent Tasks = 4

Flux NiFi > Service en masse – Prix

Nifi Flow>>Bulk Service – Price>>Bulk Elastic Search> Post Bulk Elasticsearch-Price: Configuration/Concurrent Tasks = 16
Nifi Flow>>Bulk Service – Price>>Bulk Elastic Search> Set stage.end.time: Configuration/Concurrent Tasks = 4
Nifi Flow>>Bulk Service – Price>>Bulk Elastic Search> Analyze Bulk Response: Configuration/Concurrent Tasks = 4
Nifi Flow>>Bulk Service – Price>>Bulk Elastic Search>Route On Environment: Configuration/Concurrent Tasks = 4

Flux NiFi > Service en masse – Stock

Nifi Flow>>Bulk Service – Inventory >>Bulk Elastic Search>Track Bulk Request: Configuration/Concurrent Tasks = 4
Nifi Flow>>Bulk Service – Inventory >>Bulk Elastic Search> Post Bulk Elasticsearch-Inventory: Configuration/Concurrent Tasks = 16
Nifi Flow>>Bulk Service – Inventory >>Bulk Elastic Search> Set stage.end.time: Configuration/Concurrent Tasks = 4
Nifi Flow>>Bulk Service – Inventory >>Bulk Elastic Search> Analyze Bulk Response: Configuration/Concurrent Tasks = 4
Nifi Flow>>Bulk Service – Inventory >>Bulk Elastic Search>Route On Environment: Configuration/Concurrent Tasks = 4
Nifi Flow>>Bulk Service – Inventory >>Bulk Elastic Search>Retry Flow File: Configuration/Concurrent Tasks = 4

Flux NiFi > Service NLP

Nifi Flow>>NLP Service>>Parse NLP Messages>>SCROLL Elasticsearch>>Generate Query: Configuration/Concurrent Tasks = 6
Nifi Flow>>NLP Service>>Parse NLP Messages>>SCROLL Elasticsearch>>GET Elasticsearch: Configuration/Concurrent Tasks = 6
Nifi Flow>>NLP Service>>Parse NLP Messages>>SCROLL Elasticsearch>> SCROLL Elasticsearch: Configuration/Concurrent Tasks = 6
Nifi Flow>>NLP Service>>Parse NLP Messages>>NLP Processor>>Transform Document-CoreNLP Ingest Processor: Configuration/Concurrent Tasks = 16

Nifi Flow>>NLP Service>>Parse NLP Messages>>Bulk Elasticsearch>> Track Bulk Request: Configuration/Concurrent Tasks = 4
Nifi Flow>>NLP Service>>Parse NLP Messages>>Bulk Elasticsearch>> Post Bulk Elasticsearch: Configuration/Concurrent Tasks = 16
Nifi Flow>>NLP Service>>Parse NLP Messages>>Bulk Elasticsearch>> Analyze Bulk Response: Configuration/Concurrent Tasks = 4
Nifi Flow>>NLP Service>>Parse NLP Messages>>Bulk Elasticsearch>> Retry Flow File: Configuration/Concurrent Tasks = 4

Changements de la taille du compartiment

1i

Nifi Flow> live reindex- DatabaseProductStage1i: Variables:scroll.bucket.size=5000

1e
Nifi Flow> live reindex- DatabaseProductStage1e: Variables:scroll.bucket.size=5000

Modification de l'intervalle d'actualisation.

NiFi Flow> live reindex- StoreSchema >> Setup Elasticsearch Index Schema : Populate Store Index Schema

Note: Arrêtez le processeur, cliquez sur l'objet json et éditez-le en remplaçant la valeur refresh_interval par la valeur souhaitée.