Memory Considerations for Cloud native

Defining and Inspecting Memory Settings

In Kubernetes, memory allocation for a container is controlled using two fields in the pod specification: requests and limits

• resources.requests.memory

  • Definition: The minimum amount of memory the pod is guaranteed.

  • Usage: The Kubernetes scheduler uses this value to decide where to place the pod. It will only schedule the pod on a node with enough available memory.

• resources.limits.memory

  • Definition: The maximum amount of memory the container is allowed to use.

  • Usage: If the container’s memory usage (Resident Set Size) exceeds this limit, it is terminated by the kernel with an OOMKill(Out of Memory Kill) event. You will see this as Exit Code 137.

Best Practice:

For critical, memory-sensitive applications such as JVM-based workloads, set requests.memory equal to limits.memory.

This assigns the pod a Guaranteed Quality of Service (QoS) class—ensuring it remains stable and is only terminated if it exceeds its own defined limit.

• Node Capacity: The scheduler will not place a pod on a node if the sum of the requests of all pods on that node (plus the new pod's request) exceeds the node's allocatable memory.
• Max Desired Replicas (HPA): The requests value is fundamental to the Horizontal Pod Autoscaler (HPA), which is an option for the executor pod. The HPA scales based on the percentage of requests used. Setting an accurate request is essential for proper autoscaling.

Configuring Memory in the HCL Link values.yaml

In the HCL Link Helm chart, you configure these values for each component by modifying its resources block in your values.yaml file. You then apply these changes during Helm installation or upgrade (for example helm upgrade ... -f my-values.yaml).

The default settings for most components (client, server, rest, executor) already follow the Guaranteed QoS best practice, with requests set equal to limits.

Client (client):

# values.yaml
client:
  ...
  resources:
    requests:
      memory: 4Gi
    limits:
      memory: 4Gi

Server (server)

# values.yaml
server:
  ...
  resources:
    requests:
      memory: 8Gi
    limits:
      memory: 8Gi

Rest (rest)

# values.yaml
rest:
  ...
  # 1. Set the container's total memory limit
  resources:
    requests:
      memory: 8Gi
    limits:
      memory: 8Gi

  # 2. Set the JVM's heap size *within* that limit
  #    This example sets the heap to 40% of the 8Gi limit.
  #    See the next section for why.
  jvmOptions:
    - "-XX:InitialRAMPercentage=40.0"
    - "-XX:MaxRAMPercentage=40.0"
    - "-XX:NativeMemoryTracking=summary"

Executor (executor)

# values.yaml
executor:
  ...
  resources:
    requests:
      memory: 8Gi
    limits:
      memory: 8Gi

Kafka Link (kafkaLink)

# values.yaml
kafkaLink:
  ...
  resources:
    requests:
      cpu: 250m
      memory: 4Gi
    limits:
      cpu: 2000m
      memory: 4Gi

This configuration means the pod is guaranteed only 700Mi and can "burst" up to 4Gi if memory is available on the node. However, it will be one of the first pods terminated if the node experiences memory pressure.

# Recommended production setting for kafkaLink
kafkaLink:
  ...
  resources:
    requests:
      memory: 4Gi
    limits:
      memory: 4Gi

Resource sizing as per the data volume and parallel instances

Recommended Configuration for Single-Replica Executor Flows

Recommended Configuration for Request Counts Between 250k and 1000k (Batch Size = 500)

JVM Memory: Heap, Non-Heap, and Container Limits

The Kubernetes memory limit (resources.limits.memory) applies to the entire container process—not just the JVM heap (-Xmx).

A JVM’s total memory footprint consists of:

1. JVM Heap: Holds Java objects.
2. Non-Heap Memory: Includes metaspace, thread stacks, code cache, and GC overhead.
3. Native Memory: Used by C/C++ libraries through JNI.

If the combined memory usage of these three components exceeds the Kubernetes memory limit, the pod will be terminated with OOMKilled (Exit Code 137).

Best Practice: Set JVM Heap as a Percentage

Avoid hardcoding heap size (for example, -Xmx6g). Instead, use percentage-based flags so the JVM allocates heap memory based on the container’s total memory limit.

• -XX:MaxRAMPercentage=40.0: Sets the maximum heap size (-Xmx) to 40% of the container’s memory limit, leaving around 25% for non-heap usage.
• -XX:InitialRAMPercentage=40.0: Sets the initial heap size (-Xms) to the same percentage, improving startup performance.
• -XX:NativeMemoryTracking=summary: Enables memory diagnostics to monitor JVM non-heap usage (required for the scripts below).

In the values.yaml file, configure these flags as:

• rest.jvmOptions: For the Rest pod (as an array of strings).

• rest.workerJvmOptions: For the Executor pod (as a single string).

Executor Pod: JVM + Native C++ Code

This section applies to the Executor pod, which runs both JVM and native C++ components.

The JVM flag -XX:MaxRAMPercentage manages only JVM_heap memory and reserves space for JVM_non_heap usage. It does not account for memory consumed by native C++ processes. You must factor that in manually.

K8s_Limit = Max_JVM_Heap + Max_JVM_Non_Heap + Max_C++_Memory + Buffer

To size this correctly:

1. Measure Max_JVM_Non_Heap and Max_C++_Memory under load.

2. Adjust Max_JVM_Heap (using rest.workerJvmOptions) so the total fits within the defined executor.resources.memory limit.

Scripts to Check Memory Usage (JVM vs. Native)

You can check memory usage by running these scripts inside the running executor pod. Start by connecting to the pod and finding the Java process ID (usually 1).

$ kubectl exec -it <your-executor-pod-name> -- /bin/bash
root@my-pod:/# ps aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
java           1  5.5 10.1 5456252 833308 ?      Ssl  10:00   0:35 /usr/bin/java ...

Script 1: Check JVM Internal Memory (Heap vs. Non-Heap)

Use jcmd to get the JVM’s internal memory details.

#!/bin/bash
# check_jvm_nmt.sh

# PID of the Java process
PID=1
echo "--- JVM Native Memory Tracking (NMT) ---"
jcmd $PID VM.native_memory summary

Native Memory Tracking:

Total: reserved=5075788KB, committed=1310123KB  <-- (A) Total JVM Committed Memory
-                 Java Heap (reserved=4194304KB, committed=1048576KB)
-                     Class (reserved=135444KB, committed=24460KB)
-                    Thread (reserved=22420KB, committed=22420KB)
...

The key value is (A) Total Committed: 1310123KB (~1.25Gi) — this shows total memory tracked by the JVM.

Script 2: Check Total Process Memory (JVM + C++)

Use pmap to view the process’s total memory footprint (JVM + native C++).

#!/bin/bash
# check_total_process_memory.sh

# PID of the Java process
PID=1
echo "--- Total Process Memory (pmap) ---"
pmap -x $PID | tail -n 1

total kB         5229348  1845180  1800180
                          ^
                          |
                        (B) Total RSS (Actual RAM)

The key value is (B) Total RSS: 1845180KB (~1.76Gi) — this shows the actual total memory used by the container.

Setting the Appropriate JVM Heap Size