Backing up and restoring data for Elasticsearch-based components

Back up and restore Elasticsearch-based data from the HCL Connections™ Component Pack system that contains the Kubernetes master server. Because you are connected to the Elasticsearch container, by default you have root access to run commands inside the container.

This task applies to the metrics, type-ahead search, Orient Me, and Elastic stack features of Component Pack, depending on what features are configured.

Register the snapshot repository in Elasticsearch 7 cluster and back up the index.
1. Create the snapshot:
  1. Get a shell to the running container (es-client):
```
kubectl exec -ti -n connections $(kubectl get pods -n connections | grep es-client | awk '{print $1}') -- /bin/bash
```
    Locate the current directory by running:
```
pwd
```
    This command shows the following output:
```
/opt/elsticsearch-7.10.1
```
  2. Go to the "probe" directory, that is /opt/elsticsearch-7.10.1/probe
  3. Create a snapshot repository:
```
./sendRequest.sh PUT /_snapshot/<REPO> -H 'Content-Type: application/json' -d '{"type": "fs","settings": {"compress": true, "location":  "<BACKUPPATH>"}}'
```
    Where:
    - <REPO> is the snapshot repository name, for example hclcnx_es7.
    - <BACKUPPATH> is the mount path of the Elasticsearch 7 backup persistent volume (/pv-connections/esbackup-7). By default, this path is /backup.
    For example:
```
./sendRequest.sh PUT /_snapshot/hclcnx_es7 -H 'Content-Type: application/json' -d '{"type": "fs","settings": {"compress": true, "location": "/backup"}}'
```
2. Back up the Elasticsearch 7 indices.
  1. Run the following command.
    Note that we use snapshot$(date +%Y%m%d%H%M%S) as the snapshot name, but if you need a customized one, change it accordingly.
```
./sendRequest.sh PUT /_snapshot/${REPO}/snapshot$(date +%Y%m%d%H%M%S)?wait_for_completion=true
```
    Where {REPO} is the snapshot repository name, for example hclcnx_es7.
    For example:
```
./sendRequest.sh PUT /_snapshot/hclcnx_es7/snapshot$(date +%Y%m%d%H%M%S)?wait_for_completion=true
```
    With wait_for_completion=true, this command will end when backup finishes, then the backup results will be printed. In the given example, the backup indices are created into the /backup directory, which you can view outside of the container at /pv-connections/esbackup-7.
  2. The following are additional commands that you can use:
    - To check all snapshots:
```
./sendRequest.sh GET /_snapshot/<REPO>/_all?pretty
```
      Where <REPO> is the snapshot repository name, for example hclcnx_es7.
      For example:
      ./sendRequest.sh GET /_snapshot/hclcnx_es7/_all?pretty
      The output of this command provides the snapshot name (<SNAPSHOT>), UUID, and details of indices.
    - To delete a snapshot:
```
./sendRequest.sh DELETE /_snapshot/<REPO>/<SNAPSHOT>?pretty
```
      Where:
      
      <REPO> is the snapshot repository name, for example hclcnx_es7.
      
      <SNAPSHOT> is the snapshot name.
      For example:
      ./sendRequest.sh DELETE /_snapshot/hclcnx_es7/snapshot20221019232050?pretty

Get the index list.

Get a shell to the running container (es-client):
```
kubectl exec -ti -n connections $(kubectl get pods -n connections | grep es-client | awk '{print $1}') -- /bin/bash
```
Locate the current directory by running:
```
pwd
```
This command shows the following output:
```
/opt/elsticsearch-7.10.1
```
Go to the "probe" directory, that is /opt/elsticsearch-7.10.1/probe
Run ./sendRequest.sh GET /_cat/indices
Note: If you get any green or yellow index statuses, that is fine. If you encounter any red status, fix the Elasticsearch 7 cluster first.

Record the index names for the applications that you intend to migrate. These names will be used for the rest of the migration process.

Application	Index name	Number of indices
Metrics	icmetrics_a_YYYY_{1h \| 2h} For example, icmetrics_a_2019_2h	Two per calendar year of data collection
Type-ahead search	quickresults	One

Restore Elasticsearch 7.
1. Get a shell to the running container (es-client):
```
kubectl exec -ti -n connections $(kubectl get pods -n connections | grep es-client | awk '{print $1}') -- /bin/bash
```
  Locate the current directory by running:
```
pwd
```
  This command shows the following output:
```
/opt/elsticsearch-7.10.1
```
2. Go to the "probe" directory, that is /opt/elsticsearch-7.10.1/probe
3. Before restoration, you need to close indices first. You can choose to either close all indices at once then restore a snapshot, or close a specific index then restore it afterward:
  - For all indices:
    1. Close all indices first:
```
./sendRequest.sh POST /_all/_close
```
    2. Restore a snapshot in the repository by running this command:
```
./sendRequest.sh POST /_snapshot/${REPO}/<SNAPSHOT>/_restore?wait_for_completion=true
```
      Where:
      
      {REPO} is the snapshot repository name, for example hclcnx_es7.
      
      <SNAPSHOT> is the snapshot name.
      For example:
      ./sendRequest.sh POST /_snapshot/hclcnx_es7/snapshot20221019232050/_restore?wait_for_completion=true
  - For a specific index:
    1. Close the given index first:
```
./sendRequest.sh POST /${INDEX}/_close
```
      For example, to close the Orient Me index, run this command:
      ./sendRequest.sh POST /orient-me-collection/_close
      Closing a specific index means only that index can be restored. This is because an index needs to be closed before restoration.
    2. Restore the given index:
```
./sendRequest.sh POST /_snapshot/${REPO}/<SNAPSHOT>/_restore?wait_for_completion=true -d '{"indices": "${INDEX}"}'
```
      Where:
      
      ${REPO} is the snapshot repository name, for example hclcnx_es7.
      
      <SNAPSHOT> is the snapshot name.
      
      ${INDEX} is the specific index that you closed and you want restored.
      
      For example:
      ./sendRequest.sh POST /_snapshot/hclcnx_es7/snapshot20221019232050/_restore?wait_for_completion=true -d '{"indices": "orient-me-collection"}'
4. When the restoration is complete, check the status of indices.
```
./sendRequest.sh GET /_cat/indices
```
  In the output of this command, any green or yellow index statuses is fine. If you encounter any red status, fix the Elasticsearch 7 cluster first.

If you need more information, see Snapshot module and Restoring from a Snapshot in the Elasticsearch documentation.