Adding languages to the NLP service

Natural Language Processing is enabled using the NLP_ENABLE_LANGUAGE_CODE environment variable. The user needs to set the default value to be able to use the NLP functionality. This variable is described in Environment data in Vault. To enable NLP for other supported languages, add the language code to the variable in both NIFI and Query service containers. For Docker Compose, this process is described in Deploying HCL Commerce Version 9.1 with Elasticsearch-based search and an IBM Db2 database using Docker Compose. You can also add languages in the values.yaml file, as an environment parameter (the language code is not case-sensitive):

nifiApp:
envParameters:

NLP_ENABLE_LANGUAGE_CODE:en,es

HCL Commerce NLP supports English, French, and Spanish by default. You can add German or Chinese by appending the de and zh codes to the list, for example: en,es,fr,de,zh. To enable support for more than one language, add a comma separated value from this list. For example, adding en,es will enable NLP for both English and Spanish.

Only English (code en) is supported during the initial deployment of HCL Commerce described in Deploying HCL Commerce Version 9.1 with Elasticsearch-based search and an IBM Db2 database using Docker Compose. To add an additional language, create new Docker images to accommodate the new languages for both the Ingest and Query services. Ensure that you use only Version 4.4.0 language models, as 4.4.0 is the only supported version.

For NiFi

Create the following subdirectory structure in a working directory:
Download the stanford-corenlp-4.4.0-models-*.zip file for your new language from the Release History page of the Stanford Core website, into the bundled-dependencies subdirectory.

Create the following Dockerfile in the nifi directory. Replace the sample stanford-corenlp-4.4.0-models-*.jar with the name of your downloaded language file. (Note that the server name, myserver.example.com, is an example; replace it with the address appropriate for your site.)

FROM myserver.example.com/9.1.13.1/search-nifi-app:latest

COPY --chown=nifi:nifi patches/ /patches

USER root 

RUN cd /patches && \
    jar uvf /opt/nifi/nifi-current/extensions/commerce-search-data-nar-9.1.13.0.nar META-INF/bundled-dependencies/stanford-corenlp-4.4.0-models-*.jar && \
	chmod 0755 /opt/nifi/nifi-current/extensions/commerce-search-data-nar-9.1.13.0.nar && \
	chown nifi:nifi /opt/nifi/nifi-current/extensions/commerce-search-data-nar-9.1.13.0.nar && \
    rm -rf /patches/META-INF

USER nifi

From Version 9.1.14 onwards, create the Dockerfile using the comuser user, as follows:

FROM myserver.example.com/9.1.14.0/search-nifi-app:latest

COPY --chown=comuser:comuser patches/ /patches

USER root 

RUN cd /patches && \
    jar uvf /opt/nifi/nifi-current/extensions/commerce-search-data-nar-9.1.14.0.nar META-INF/bundled-dependencies/stanford-corenlp-4.4.0-models-*.jar && \
	chmod 0755 /opt/nifi/nifi-current/extensions/commerce-search-data-nar-9.1.14.0.nar && \
	chown comuser:comuser /opt/nifi/nifi-current/extensions/commerce-search-data-nar-9.1.14.0.nar && \
    rm -rf /patches/META-INF

USER comuser

Run the following command:
```
docker build -t myserver.example.com/9.1.13.1/search-nifi-app:custom-image.
```
A new Docker NiFi image will be created that includes the new language file.

For the Query service

In a working directory, create a directory named query, and inside that directory, a subdirectory named patches.
Download the stanford-corenlp-4.4.0-models-*.zip file for your new language from the Release History page of the Stanford Core website, into the patches subdirectory.
Create the following Dockerfile in the Query directory. Replace the sample stanford-corenlp-4.4.0-models-*.jar with the name of your downloaded language file.
```
FROM myserver.example.com/9.1.13.1/search-query-app:latest

COPY patches/stanford-corenlp-4.4.0-models-*.jar /profile/patches

RUN chmod -R 0755 /SETUP/bin && \
   /SETUP/bin/patch.sh
```
Run the following command:
```
docker build -t myserver.example.com/9.1.13.1/search-query-app:custom-image.
```
A new Docker Query service image will be created that includes the new language file.