Google BigQuery Connector
This topic describes how to integrate Google BigQuery and HCL CDP using the HCL Link connector to retrieve and transform flattened event data.
Introduction
The BigQuery adapter retrieves flattened event data from BigQuery, including core and event-specific fields. Data from Google Analytics 4 (GA4) is automatically ingested into BigQuery with a one-day delay (T-1), with separate tables created for each day. The adapter transforms this data into the DI API request format for HCL CDP.
Prerequisites
Before you begin, ensure you have the GCP (Google Cloud Platform) Service JSON key file used to authenticate for Google BigQuery.
Connection Parameters
The administrator must configure the following connection properties to establish communication:
| Property | Description |
|---|---|
| CDP API URL | Base URL for CDP DI API endpoint (Example: <https://<base_url>>/v3/data). |
| CDP API Key | CDP DI API key used for authentication (x-api-key). |
| CDP API Pass Key | CDP DI API passkey used for additional authentication (x-api-passkey). |
| Service Filename | A JSON key file for secure access. Important: The file should be placed on the LINK
deployed server at
/opt/hip-rest. |
| Fetch Units | Units used to schedule the flow (e.g., hours, minutes). |
| Fetch Interval | The specific interval value to schedule the flow. |
Action Properties
Configure the following properties when creating an action for the GBQ Event Data connector:
| Property | Description |
|---|---|
| Google Cloud Project ID | Project ID where the Google BigQuery resources are stored. |
| GA4 Property ID | Unique number representing your GA4 setup to link with BigQuery. |
| CDP Primary Identifier | Primary unique identifier for HCL CDP. |
| Contextual Metadata | Main event data fields (core fields). Parameters like
user_pseudo_id, event_name,
and event_timestamp are handled automatically.
Additional parameters (e.g., platform,
geo.city) must be comma-separated. |
| Event Parameters | Additional key-value pairs (e.g., page_location,
click_text) providing context. These must be
comma-separated. |
| Filter by Event | Comma-separated event names to filter data. If blank, data for all events is fetched. |
| Distributor Max Instances | Specifies the maximum number of distributed batch instances that can run in parallel. |
| Distributor Node Batch Size | Defines the number of records to process in each distributed batch. |
| Duration | Number of days the flow will run. |
Known Issues and Resolution
| Known Issue | Resolution |
|---|---|
| 403 Forbidden error on "Test Connection" (Invalid API Key or Pass Key). | Ensure you are using a valid CDP URL and the correct pass key associated with the tenant. |