Azure Blob Storage Connector Reference

The Azure Blob Storage Connector enables seamless data synchronization by providing guidelines for using the Azure Blob Storage adapter to retrieve user behavior data from multiple files and securely send them to HCL CDP (Customer Data Platform).

Introduction

The Azure Blob Storage Connector is designed to connect to your Azure Storage account, retrieve user behavior data natively stored in CSV files, and push this data into HCL CDP for enhanced user tracking and analytics. The connector relies on mandatory fields such as user identification (e.g., user_id or customer_id) and event names to accurately process the data.

Prerequisites

  • Azure Storage Account: A valid Microsoft Azure Storage account where your files are securely stored.
  • Azure Storage Account Access Key: The authentication key or connection string required to securely access the blob storage. This can be generated via the Microsoft Azure Portal under Security + networking > Access keys.

Connection Box

The CDP administrator must configure the following connection properties:

Table 1. Connection Box Properties
Property Description
Azure Storage Account Name The name of the Azure storage account where the files are stored.
Azure Storage Access Key The key used to authenticate with your Azure Storage account.
CDP API URL The CDP API endpoint URL (e.g., https://crux.dev.hxcb.now.hclsoftware.cloud/v3/data).
CDP API Key The API key used to authenticate the CDP tenant (x-api-key).
CDP API Pass Key The API pass key used to authenticate the CDP tenant (x-api-passkey).
Activity Fetch Units The fetch interval to schedule the flow to run (based on hours, minutes, or seconds).
Activity Fetch Frequency The fetch interval time value (e.g., 2).
Number of Retries The number of times to retry the API call if the server is inaccessible.

Action Box Properties

Define the following properties when configuring the connector from an action box:

Table 2. Action Box Properties
Action Box Property Description
Azure Storage Container Name Specifies the name of the Azure Blob Storage container where the files are located.
Input Files Type Select the file type from the drop-down menu (CSV).
User ID Field Name The name of the user ID column/key. This is a mandatory field.
EventName Field Name The name of the event name column/key. This is a mandatory field.
CDP Primary Identifier The primary identifier name for CDP (e.g., identity) accepted based on the tenant configuration.
Duration (in days) The duration in days that the flow should run.
Maximum Instances The number of instances to run in parallel. Default is 1.
Batch Size The batch size to send to CDP. Default is 500.

Limitations

The system currently supports processing up to 100,000 records per file for CSV formats of max 100MB size per file. It is recommended to split larger datasets into multiple files to ensure successful processing.

Event Tracking and Examples

The response from the Azure Blob Storage connector provides tracking events to monitor data ingestion success or failure. The payload contains the following fields: status_message, status_code, message, id, and timestamp.

Example API Details

Method: POST
                URL: https://crux.dev.hxcb.now.hclsoftware.cloud/v3/data
                Headers: 
                x-api-key: <api-key-value>
                x-api-passkey: <pass-key-value>

Success Scenario Example

"status_message","statusCode","message","id","timestamp"
"Success","200","Data Processed successfully","ebc2bedf-5535-4a1f-adaf-ee60d0ff3790","2025-04-03T12:20:56"

Error Scenario Example

"status_message","statusCode","message","id","timestamp"
 "Error","500","Internal server error",,"2025-09-17 13:24:46"

Known Issues and Resolutions

Known Issues Resolutions
FILE NOT EXISTS. The source is not available. Check the Azure file name and container, and provide the correct location.
403 Forbidden. The server failed to authenticate the request due to an invalid Storage Access Key. Verify the access key from the Microsoft Azure Portal and regenerate the key if needed.
404 Resource Not Found. The specified resource does not exist, likely caused by a wrong container name or file path. Verify the storage account name, container name, and blob path. Ensure the file exists in the targeted container.
500 / 503 Server Errors. The server is busy or encountered an internal error due to temporary Azure platform issues. Implement retry logic and retry the execution after a short delay.