Persisting Ingest changes (NiFi Registry)
This topic describes how to make changes to a specific pipe by changing its property/configuration across all connectors. For example, changing SQL statement for a processor, Execute SQL, in a pipe, DatabaseCategoryStage1b, across all connector in NiFi.
About this task
Procedure
-
Find the processor that you want to change.
Note: You can make only property/configuration changes and no logic change to the processor.
-
Find all the pipes in NiFi containing the processor that you want to change and
then make a list of
pipe.name. -
Run the following command to navigate to NiFi Registry container/pod.
docker exec -it {registryContainer} bash -
Run the following command to extract the pipe description JSON for the list of
pipe.namecreated in Step 2.sh /opt/nifi-registry/scripts/export_flow.sh {pipe.name} > /opt/nifi-registry/{pipe.name}.json -
Open {pipe.name}.json and then search for the desired processor for editing and
changing its property/configuration value.
Ensure that you update the timestamp in the snapshotmetadata of the JSON file to the current timestamp. If this is not done, the changes will not be recognized.
{ "snapshotMetadata": { "version": 3, "timestamp": 1722625593, "author": "anonymous", "comments": "Build Level: 9.1.15.1/search-registry-app=v9-20240115-1742" } }The successful file import results in the following log entry:INFO [import] Current version of DatabaseProductStage1a flow has an earlier timestamp and needs to be upgraded: 1722539700 vs 1722625593
INFO [import] Imported a newer version from /opt/nifi-registry/DatabaseProductStage1a.json into DatabaseProductStage1a flow as version 3
-
Run the following command to import the updated pipe description JSON for the
list of
pipe.name.sh /opt/nifi-registry/scripts/import_flow.sh {pipe.name} /opt/nifi-registry/{pipe.name}.json -
Run the following command to exit out of the container and backup the updated
pipe description JSON.
docker cp {registryContainer}:/opt/nifi-registry/{pipe.name}.json {pipe.name}.json - Using the following URL, check NiFi Registry to verify that these pipe description JSON (flow) have a new version defined.
-
Use any one of the following two ways (a or b) to update the pipes in NiFi with
a new version: