Customizing Search for CI/CD automation

Use the following guidelines to customize your code in such a way that your changes can be automatically published via a Continuous Integration/Continuous Delivery (CI/CD) process.

Create search customizations

The Commerce Search service includes Ingest service and Elasticsearch, which together form a distributed, scalable search. Ingest service uses a configurable technology called NiFi, and its custom connector pipes. You can extend NiFi by:

Creating a custom NiFi processor

Here is the list of tasks that the developers need to perform for customizing NiFi processors

Create a custom NiFi processor. You can add a new processor to modify the Ingest connector flow/pipeline. A new processor can be created writing a new logic in Java or any other programming language. For more details, refer to Commerce+ Transaction Server Developer Elasticsearch-based Search NiFi environment
Create the jUnits to test the custom NiFi processors.
Using the JSON with mocked up input flow data, ensure that the custom processors work as expected.
Check-in all the resources in the GIT Repo, under the directory /nifi/processor/*
Note: You can create multiple processors using one NAR file.

A sample GIT repository structure for this component

SearchCommerce+
├── nifi
│   ├── README.md
│   └── processor
│       └── commerce-custom-search-bundles
│           ├── commerce-custom-search-processors
│           │   └── pom.xml
│           ├── commerce-custom-search-processors-nar
│           │   └── pom.xml
│           └── pom.xml

Creating a custom process group

Here is the list of tasks that the developers need to perform for customizing NiFi process groups

Create a custom process group. You can add a new process group to modify the Ingest connector flow/pipeline. A new process group can be created by using the default templates from the NiFi registry. For more details, refer to Creating a custom NiFi process group. The most common custom process group is the process group created to modify the Elasticsearch schema.
Using the NiFi UI perform the following tasks:
1. Stop the version control for the desired process groups.
2. Start the version control for the desired process groups.
3. Provide a unique name for the process groups.
4. Connect to the NiFi Registry.

Creating a custom Ingest connector

Here is the list of tasks that the developers need to perform for creating new Ingest connectors and NiFi pipeline.

Create a new Ingest connector by using the NiFi runtime UI. The Ingest connector is the connections between the process groups that make up a NiFi Pipeline. Adding a new connector implies the addition of a new NiFi pipeline and configuration that resides in the zookeeper and NiFi Registry. It involves extending the default Ingest Connector (for example, auth.reindex) and pipelines that exist in NiFi.
Modify the process groups.
Test the custom process groups by using the NiFi UI.
Retrieve the connector descriptor using the Ingest container API end point :
```
curl -X GET "http://v91tk.hclcdev.com:30800/connectors/auth.reindex" -H "accept: application/json"
```
Modify the Ingest connector descriptor using a text editor.
Check in the modified Ingest Connector definition into a GIT repository.
A sample GIT repository structure for this component:
```
SearchCommerce+
├── ingest
│   ├── README.md
│   └── connector
│       └── auth_reindex.json
│       └── pushtolive.json
```
Upload the modified Ingest connector using the Ingest service APIs to persist the changes in ZooKeeper.

Note: Refer to the Create the customization in NiFi to familiarize with customization steps.