Sample steps to upgrade and migrate data from Component Pack 6.5 to 7

Before you begin

For background information and where to get the latest download, see Installation and upgrades.

Before we start with the sample steps, let's make a few assumptions about the environment.

We are upgrading an environment which will be available behind web1.cnx-dev.net. However, this is just a public domain, and entry point to the Connections (and dynamicHost of Connections).

All the nodes are available using their FQDNs and are in a different domain:

• Connections, with WebSphere and IBM HTTP Server is on connections1.internal.example.com
• DB2 is on db1.internal.example.com
• Kubernetes cluster is just a single node, and it is on cp1.internal.example.com
• NFS master is collocated with cp1.internal.example.com. Its IP address is 172.27.1.48 and all your folders are created in 172.27.1.48:/pv-connections/

About this task

This example uses the preceding assumptions to walk you through the following steps, in a logical order, to get your Component Pack deployment up and running:

• Set up NFS
• Download and set up the package
• Push the images to the Docker Registry or Amazon ECR
• Use –f instead of –set with Helm charts
• Set up persistent volumes and persistent volume claims on NFS
• Set up bootstrap charts
• Set up connections-env charts
• Set up infrastructure charts
• Set up Customizer
• Set up Elasticsearch 7 charts
• Set up Microsoft Teams integration
• Set up Tailored Experience features for communities
• Set up Orient Me for Elasticsearch 5
• Migrate data from Elasticsearch 5 to 7
• Verify migration
• Switch the indexing service from Elasticsearch 5 to 7
• Uninstall Elasticsearch 5
• Enable Metrics for Elasticsearch 7
• Upgrade Helm from v2 to v3
• Set up ingress
• Set up Activities Plus
• Set up Connections Add-in for Microsoft Outlook
• Enable Microsoft Teams integration
• Enable Connections Add-in for Microsoft Outlook

Set up NFS

• Create /pv-connections folder on cp1.internal.example.com
• Inside that folder, create this set of subfolders:
  • /pv-connections/customizations with permissions 0005
  • /pv-connections/esbackup-7 with permissions 0700 (Inside /pv-connections, create a subfolder for Component Pack upgrade )
  • /pv-connections/esdata-7-0 with permissions 0700 (Inside /pv-connections, create a subfolder for Component Pack upgrade )
  • /pv-connections/esdata-7-1 with permissions 0700 (Inside /pv-connections, create a subfolder for Component Pack upgrade )
  • /pv-connections/esdata-7-2 with permissions 0700 (Inside /pv-connections, create a subfolder for Component Pack upgrade )
  • /pv-connections/esmaster-7-0 with permissions 0700 (Inside /pv-connections, create a subfolder for Component Pack upgrade )
  • /pv-connections/esmaster-7-1 with permissions 0700 (Inside /pv-connections, create a subfolder for Component Pack upgrade )
  • /pv-connections/esmaster-7-2 with permissions 0700 (Inside /pv-connections, create a subfolder for Component Pack upgrade )
  • /pv-connections/kudos-boards-minio with permissions 0700
  • /pv-connections/mongo-node-0 with permissions 0700
  • /pv-connections/mongo-node-1 with permissions 0700
  • /pv-connections/mongo-node-2 with permissions 0700

Ensure that all of the preceding folders are exported and mountable from your Kubernetes workers before you proceed.

Download and set up the package

On the server which has Helm v2 and kubectl configured for your non-root user, download and unpack the Component Pack 7.0.0.1 archive to /opt/microservices_connections.

Push the images to the Docker Registry or Amazon ECR

Inside your Component Pack package, you basically get a set of Docker prebuild images and Helm charts. You need to push those images now to a Docker Registry, which in this example lives on cp1.internal.example.com on port 5000, has SSL enabled, and can be logged in to using the username admin and password password (whatever you set for them):

• Go to component_pack_installation_folder/hybridcloud/support
• Run this command: ./setupImages.sh -dr cp1.internal.example.com:5000 -u admin –p password

For an upgrade, the script adds only the new packages, and Helm charts know which version they need. Once this finishes, you should be good to go.

The same script is compatible with Amazon ECR.

Use –f instead of –set with Helm charts

An examples folder (provided on the HCL Support site) follows the naming convention that is used throughout this topic. Feel free to check the examples for what gets rewritten for each chart.

This change makes it much cleaner, and is also the way that the HCL-provided Ansible automation works. If you want to review or customize the examples, here's how:

• Go to the Component Pack installation files for Helm chart configuration Knowledge Article on the HCL Software Support site, and download and unzip the folder cnx7-CP-examples.zip.
  Important: For the required Connections 7.0 update that the Component Pack 7.0.0.1 is part of, replace this zip even if you already downloaded it when you installed or upgraded to 7.0, as some settings changed in 7.0.0.1.
• Copy the files to component_pack_installation_folder/hybridcloud/examples.
• In the examples folder, open the subfolder that applies to your environment (either single or multiple domains).
• Review the examples, or customize them by changing the values to the ones for your environment.
• Continue with the steps that follow in this document.

Set up persistent volumes and persistent volume claims on NFS

helm upgrade connections-volumes component_pack_installation_folder/hybridcloud/helmbuilds/connections-persistent-storage-nfs-*.tgz -i -f 
component_pack_installation_folder/hybridcloud/examples/multi_domain_environment/connections-volumes.yml

Set up bootstrap charts

Bootstrap is one-time job. It creates secrets and certificates for various components, including Redis, MongoDB and ElasticSearch.

Be aware that simply rerunning/upgrading bootstrap also means recreating all those secrets and certificates, and requires in some cases redoing the steps which were dependent on it (like setting up certificates again for Metrics).

Starting with Component Pack 7, bootstrap will also create secrets and certificates for ElasticSearch 7, overwriting existing secrets so that Elasticsearch upgraders will later need to redo some configuration steps, like SSL interoperability.

helm del --purge bootstrap 
helm upgrade bootstrap component_pack_installation_folder/hybridcloud/helmbuilds/bootstrap*.tgz -i -f 
component_pack_installation_folder/hybridcloud/examples/multi_domain_environment/bootstrap.yml

Set up connections-env charts

This chart by default installs connections-env configmap.

The configmap for connections-env contains all the variables needed for the Customizer and Orient Me to function properly. Note that Customizer always points to the IBM HTTP Server directly, whereas Orient Me requests point to the front door proxy.

helm upgrade connections-env 
component_pack_installation_folder/hybridcloud/helmbuilds/connections-env-*.tgz -i -f 
component_pack_installation_folder/hybridcloud/examples/multi_domain_environment/connections-env.yml

Set up infrastructure charts

This will install the infrastructure for OrientMe and other apps, the most prominent being MongoDB and appreg services.

If this step fails, and if all pods don't come up, there is no point proceeding until this is fixed. To troubleshoot the Component Pack installation check out the Troubleshooting Component Pack document.

helm upgrade infrastructure component_pack_installation_folder/hybridcloud/helmbuilds/infrastructure-*.tgz -i -f 
component_pack_installation_folder/hybridcloud/examples/multi_domain_environment/infrastructure.yml

Set up Customizer

helm upgrade mw-proxy 
component_pack_installation_folder/hybridcloud/helmbuilds/mw-proxy-*.tgz -i -f 
component_pack_installation_folder/hybridcloud/examples/multi_domain_environment/customizer.yml

location ~      ^/(files/customizer|files/app|communities/service/html|forums/html|search/web|homepage/web|social/home|mycontacts|wikis/home|blogs|news|activities/service/html|profiles/html|viewer)  { 

            proxy_pass http://cp1.internal.example.com:30301; 

}

Customizer will not start serving any traffic until you start sending some, from whatever fronted reverse proxy you are using.

Learn more about configuring Customizer in Configuring the Customizer component.

Set up Elasticsearch 7 charts

With Connections 7, ElasticSearch 7 is the default backend for Orient Me. It is also the default backend for typeahead search and Metrics.

Solr and Zookeeper are not used anymore, so feel free to clean them up if you didn't already.

helm upgrade elasticsearch7 
component_pack_installation_folder/hybridcloud/helmbuilds/elasticsearch7-*.tgz -i -f 
component_pack_installation_folder/hybridcloud/examples/multi_domain_environment/elasticsearch.yml

Set up Tailored Experience features for communities

A replacement strategy is being used to install or upgrade the Tailored Experience features; therefore it's important to note that you must add --force to the end of the command.

helm upgrade tailored-exp 
component_pack_installation_folder/hybridcloud/helmbuilds/tailored-exp-*.tgz -i -f 
component_pack_installation_folder/hybridcloud/examples/multi_domain_environment/tailoredexperience.yml --force

# proxy rules for admin-portal
ProxyPass "/cnxadmin" "http://cp1.internal.example.com:32080/cnxadmin"  
ProxyPassReverse "/cnxadmin" "http://cp1.internal.example.com:32080/cnxadmin"
# proxy rules for community-template-service
ProxyPass "/comm-template" "http://cp1.internal.example.com:32080/comm-template/templates" 
ProxyPassReverse "/comm-template" "http://cp1.internal.example.com:32080/comm-template/templates"
# proxy rules for te-creation-wizard
ProxyPass "/te-creation-wizard/" "http://cp1.internal.example.com:32080/te-creation-wizard/" 
ProxyPassReverse "/te-creation-wizard/" "http://cp1.internal.example.com:32080/te-creation-wizard/"

Set up Orient Me for Elasticsearch 5

With Connections 7, the default and only backend for Orient Me is ElasticSearch, and in the provided yaml files, the default Elasticsearch version is 7.

helm upgrade -i orientme orientme*.tgz --set 
global.onPrem=true,global.image.repository=localhost:5000/connections,orient-web-client.service.nodePort=30001,itm-services.service.nodePort=31100,mail-service.service.nodePort=32721,community-suggestions.service.nodePort=32200,orient-indexing-service.indexing.solr=false,orient-indexing-service.indexing.Elasticsearch=true,orient-indexing-service.indexing.Elasticsearch7=false,orient-retrieval-service.retrieval.Elasticsearch=true

Migrate data from Elasticsearch 5 to 7

Verify migration

Switch the indexing service from Elasticsearch 5 to 7

To complete Orient Me set up, you also need to do the following:

• After you set up Orient Me to point at Elasticsearch 7.6.1, you need to run profiles migration. In this step, we are migrating users from PeopleDB to MongoDB, which is used by Orient Me.
• You need to set rewrite rules in httpd.conf on your IBM HTTP Server to enable sending requests to it from Connections.

helm upgrade orientme 
component_pack_installation_folder/hybridcloud/helmbuilds/orientme-*.tgz -i -f 
component_pack_installation_folder/hybridcloud/examples/multi_domain_environment/orientme.yml

kubectl exec -n connections -it $(kubectl get pods -n connections | grep people-migrate | awk '{print $1}') -- sh -c "npm run start migrate"

If you followed examples and the order of installation, this should work out of the box. If you want to do some customizations, or if you are later changing the database, you can SSSH into people-migrate pod and see the configuration that was precreated for you during the installation/upgrade of Component Pack in /usr/src/app/migrationConfig.

# OrientMe Config 
ProxyPreserveHost On 
ProxyPass "/social" "http://cp1.internal.example.com:32080/social" 
ProxyPassReverse "/social" "http://cp1.internal.example.com:32080/social" 
ProxyPass "/itm" "http://cp1.internal.example.com:32080/itm" 
ProxyPassReverse "/itm" "http://cp1.internal.example.com:32080/itm" 
ProxyPass "/community_suggestions/api/recommend/communities" "http://cp1.internal.example.com:32080/community_suggestions/api/recommend/communities" 
ProxyPassReverse "/community_suggestions/api/recommend/communities" "http://cp1.internal.example.com:32080/community_suggestions/api/recommend/communities" 
ProxyPass "/appreg" "http://cp1.internal.example.com:32080/appreg/" 
ProxyPassReverse "/appreg" "http://cp1.internal.example.com:32080/appreg/" 
ProxyPass "/appregistry" "http://cp1.internal.example.com:32080/appregistry" 
ProxyPassReverse "/appregistry" "http://cp1.internal.example.com:32080/appregistry"

Later, you can learn more about configuring Orient Me in cp_config_om_intro.html.

Uninstall Elasticsearch 5

helm delete -purge Elasticsearch

Enable Metrics for Elasticsearch 7

Upgrade Helm from v2 to v3

Now, upgrade to Helm v3, as some of the components remaining to be upgraded to Component Pack 7 require Helm 3.

For guidance on upgrading to Helm 3, see the knowledge article KB0090051: Upgrading from Connections 6.5 to 7 Whitepaper, from which you can download a PDF of the paper.

Set up ingress

Connections 7 supports only the community Nginx Ingress controller.

With community ingress, besides its being more flexible with AWS and OpenShift, you get out-of-the-box Prometheus exporters, which come in handy if you decide to also set up monitoring.

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx 

helm repo update 

helm upgrade cnx-ingress -i ingress-nginx/ingress-nginx --namespace connections --set 
"controller.service.type=NodePort,controller.service.nodePorts.http=32080,controller.service.nodePorts.https=32443"

helm upgrade cnx-ingress-rules 
component_pack_installation_folder/hybridcloud/helmbuilds/cnx-ingress-*.tgz -i -f 
component_pack_installation_folder/hybridcloud/examples/multi_domain_environment/communityingress.yml

Set up Microsoft Teams integration

The Microsoft Teams integration microservices rely on a configmap (integrations-msteams-env) and secret (ms-teams-secret) that are part of the overall connections-env deployment. They are not installed by the connections-env chart by default; the expectation is that they are enabled as needed by customers who use Microsoft Teams and wish to enable the use cases that these micro-services support.

To upgrade the connections-env deployment to enable the correct settings for Microsoft Teams, three specific pieces of information are required which should have been created and noted while performing the steps in Configuring an Azure app to support the Microsoft Teams app.

helm upgrade connections-env component_pack_installation_folder/hybridcloud/helmbuilds/connections-env-*.tgz --reuse-values --set integrations.msteams.enabled=true,integrations.msteams.tenant.id="your_tenant_id",integrations.msteams.client.id="your_bot_id",integrations.msteams.client.secret="your_bot_secret",integrations.msteams.auth.schema="3",integrations.msgraph.client.id="not_used",integrations.msgraph.client.secret="not_used",integrations.msgraph.redirect.uri="not_used",integrations.msteams.share.ui.files.api="/files/basic/api",integrations.msteams.redirect.uri="not_used",integrations.msgraph.secret.name="not_used",integrations.msgraph.auth.endpoint="not_used",integrations.msgraph.meta.endpoint="not_used",integrations.msgraph.authorize.endpoint="not_used",integrations.msgraph.token.endpoint="not_used",integrations.msteams.share.service.endpoint="my.connections.server.com",imagePullSecretName="myregkey"

Once the configmap and secret are created, continue to install the microservices that rely on them for configuration.

helm upgrade teams 
component_pack_installation_folder/hybridcloud/helmbuilds/teams-*.tgz -i -f 
component_pack_installation_folder/hybridcloud/examples/multi_domain_environment/teams.yml

Once the microservices are installed and running, add the following rules to your httpd.conf on your IBM HTTP Server and restart the server.

#teams-tab-ui
ProxyPass "/teams-tab" "http://master_node_host_name:32080/teams-tab" 
ProxyPassReverse "/teams-tab" "http://master_node_host_name:32080/teams-tab"    
# teams-tab-api 
ProxyPass "/teams-tab/api" "http://master_node_host_name:32080/teams-tab/api" 
ProxyPassReverse "/teams-tab/api" "http://master_node_host_name:32080/teams-tab/api"
# teams-share-service 
ProxyPass "/teams-share-service" "http://master_node_host_name:32080/teams-share-service" 
ProxyPassReverse "/teams-share-service" "http://master_node_host_name:32080/teams-share-service"
# teams-share-ui 
ProxyPass "/teams-share-ui" "http://master_node_host_name:32080/teams-share-ui" 
ProxyPassReverse "/teams-share-ui" "http://master_node_host_name:32080/teams-share-ui"

# Teams SameSite Fix
# Add SameSite property to all server-side set-cookie response headers
Header edit Set-Cookie ^(.*)$ "$1; SameSite=None;Secure"

# Fix for Embedded Experiences content loading in Teams Tab iframe
<Location /connections/opensocial/gadgets/ifr>
Header unset X-frame-Options
Header always set X-frame-Options "ALLOW-FROM my.connections.server.com"
</Location>

The proxy pass statements map URI to the ingress controller service to route requests to the appropriate Microsoft Teams-related micro-services. Given tightening security of more recent versions of browser, especially Chrome (or Chromium-based browsers) and the use of embedded iFrames by Microsoft Teams, passing cookies correctly between the environments requires the SameSite=None and Secure property.

Lastly, and again for security reasons, you must change the x-frame-options header to support the way that Microsoft Teams uses an iFrame to embed application content in the tabbed pages. Since Connections also uses an iFrame to display embedded experiences content, this becomes an iFrame within an iFrame. The embedded experiences content cannot be displayed if SAMEORIGIN is used because the Teams UI and Embedded Experiences content do not share a common origin host. Using ALLOW-FROM permits the header to tell the browser that it is allowed to render content in the iFrame when processing the /connections/opensocial/gadgets/ifr request from an alternative host.

Set up Activities Plus

helm upgrade kudos-boards-cp 
component_pack_installation_folder/hybridcloud/helmbuilds/kudos-boards-cp-2.*.tgz -i -f 
component_pack_installation_folder/hybridcloud/examples/multi_domain_environment/kudosboards.yml

# proxy rules for activities plus 
RewriteRule ^/activities/service/html/(.*)$ /boards/activities/service/html/$1 [R] 
ProxyPass "/boards" "http://cp1.internal.example.com:32080/boards" 
ProxyPassReverse "/boards" "http://cp1.internal.example.com:32080/boards" 
ProxyPass "/api-boards" "http://cp1.internal.example.com:32080/api-boards" 
ProxyPassReverse "/api-boards" http://cp1.internal.example.com:32080/api-boards"

Starting with version 7, be sure that you have websockets enabled on your front proxy server.

location /api-boards/ { 

                    proxy_pass https://connections1.internal.example.com; 

                    proxy_http_version 1.1; 

                    proxy_set_header Upgrade $http_upgrade; 

                    proxy_set_header Connection "upgrade"; 

                }

Find more about Activities Plus in the section Integrating with Activities Plus.

Set up Connections Add-in for Microsoft Outlook

helm upgrade connections-outlook-desktop 
component_pack_installation_folder/hybridcloud/helmbuilds/connections-outlook-desktop-*.tgz -i -f 
component_pack_installation_folder/hybridcloud/examples/multi_domain_environment/outlook-addin.yml

# proxy rules for outlook add-in
Redirect "/outlook-addin" "/outlook-addin/" 
ProxyPass "/outlook-addin/" "http://cp1.internal.example.com:31810/" 
ProxyPassReverse "/outlook-addin/" "http://cp1.internal.example.com:31810/"

Finally, update the add-in docker environment variables.

These are located in the outlook-addin.yaml file. These are passed into the Outlook add-in docker instance on startup:

• What must be overriden:
  • CONNECTIONS_URL - URL of your Connections environment without a trailing slash (eg https://my.connections.server.com). The same URL has to be used when generating secret in the first step.
  • CONNECTIONS_CLIENT_SECRET - Client secret generated by Connections when registering OAuth provider in the first step.
  • CONNECTIONS_CLIENT_ID - Client ID (aka. app ID) used when registering OAuth provider in Connections in the first step (default: hcl-cnx-office-addin)
• What may be overriden:
  • CONTEXT_ROOT - The path to where the Outlook add-in is being served, relative to the CONNECTIONS_URL. Do NOT start or end with `/. (default: outlook-addin)
  • SUPPORT_URL - URL that a user can go to for support (help). (default: https://help.hcltechsw.com/connections/v7/connectors/enduser/c_ms_plugins_add_in_outlook.html)
  • CONNECTIONS_NAME – A custom name for the add-in..(default: 'HCL Connections')
• Take care about ingresses listed there. You should point to both frontend domain and internal domains, if both are used. Otherwise, only point to the one that is used in your case.

Enable Microsoft Teams integration

See ../../connectors/admin/t_ms_teams_set_up_conn_app_for_ms.html in the Integrating with Other Products section of this documentaion.

Enable Connections Add-in for Microsoft Outlook

See Making the Connections Add-in for Outlook available to users.