Server microservices

The HCL Universal Orchestrator architecture is based on microservices, which communicate over Rest APIs and Kafka topics. These are independent services that serve specific purposes.

Each microservice covers specific roles, provides specific functionalities and persists data in a noSQL database. All microservices are active/active and can scale indefinitely horizontally (also with the auto scale feature) to dynamically adjust to the current workload. This makes HCL Universal Orchestrator always available, easy to scale, fast in running workloads and with a very low maintenance.

You can find the following information about microservices:

Cloud task launcher

The Cloud task launcher executes jobs that call external systems, like RESTful APIs, web services and databases, directly from the engine, without using an external machine. It supports auto-scaling and the high availability features of other microservices.

Differently from a dynamic agent, the Cloud task launcher runs on a separate microservice pod and does not affect server functionalities like planning, and even more importantly it does not impact the HCL Universal Orchestrator engine performances. This results in a sensible improvement on the average job execution timing.

Almost all plug-ins that run on a dynamic agent can run on the Cloud task launcher. Some of the most notable examples include:
  • RESTful APIs
  • Remote Command
  • Kubernetes
Plug-ins with access to the filesystem have that functionality removed when running on the Cloud task launcher, increasing the system security.

Executable plug-ins and native plug-ins are not supported.

Agent manager

Agent Manager microservice is the communication endpoint used by Cloud task launcher and agents to communicate with HCL Universal Orchestrator. The agent manager communicates via HTTPS and uses JWT tokens for authentication. The Cloud task launcher/agent communicates with agent manager to retrieve the job submissions, user commands and send back the changes in job status and command results.

The agents need to be configured via gateway to communicate with the Agent manager. When the Agent or Cloud task launcher is not connected, the Agent Manager keeps the pending commands and job submissions for that Agent or Cloud task launcher in the database, until they can reconnect and retrieve them.

Gateway

Gateway is the single entry point where you can provide the request to initiate the workflow. It acts as a bridge between you and a collection of microservices. The gateway will redirect the request to call up the required microservices and deliver the response back to you. It standardizes and centralizes the delivery of services through microservices.

Audit

In microservice architecture, it is critical to monitor the actions performed on items and different microservices. To ensure the security of data, Audit microservice will store the records of all the events and activities that are performed in HCL Universal Orchestrator. All the operations, such as create, read, update, and delete that are performed on the items are recorded. There is one exception, the internal activities that are performed by HCL Universal Orchestrator are not recorded.

Audit microservice helps to track all the versions of the items by navigating through the history and enables the user to analyse the changes made over time. The data stored in databases can be analysed to determine if any policies or security measures are violated.

Identification, Authentication and Authorization (IAA)

The IAA microservice authenticate or authorize the actions you can perform in HCL Universal Orchestrator and stores security related resources. The microservice stores information such as permissions or roles assigned to each ID and information of API keys and resource folders. When you log into HCL Universal Orchestrator with valid credentials, the microservice identifies the permissions associated with that ID and let you perform the actions accordingly.

Orchestration

The Orchestration microservice is a component that manage the workflow management. The microservice will store the job definitions and job stream definitions, then plan and drive the workflow. Orchestrator is also responsible for organizing the completion of different jobs. It is designed to distribute the work between multiple nodes, to ensure high availability.

As the name suggests, it orchestrates the timing to complete the workflow and resolve dependencies. The orchestrator will also store jobs and job streams progress.

Scheduler

The scheduler microservice organize the submission of jobs and job streams by assigning the available resources. The microservice identifies the agents that are down and re-initiate the workflow when they are up and running again. When the agents are down, the data about jobs and job streams are stored by the scheduler temporarily. When the agents are back online, the scheduler notifies the agent manager to re-initiate the workflow.

Storage

In HCL Universal Orchestrator, it is the storage microservice which manages the files that are required by other microservices. The job plugins (jar files) that are used by the Agent manager and API gateway are stored by this microservice. The plugin jar files are then distributed to all components/microservices that requires them. With this microservice, you can run a service without any persistent storage in Kubernetes.

Timer

The timer microservice acts as a clock in Maestro. The microservice sent messages to other microservices, to start the jobs or job streams as scheduled.

Toolbox

The toolbox microservice is used to store information about calendars and variable tables that you create or update. The microservice allow all types of operations on calendars and variable tables, such as create, read, update, and delete.