Feed Data Model

This page outlines how feed data models in HCL Detect define the structure and attributes of real-time data feeds. It explains model definitions, required fields, associated profiles, and how to configure and organize them for system integration.

Feed data models represents common classes of feed data sources in the system. E.g., Card Transaction feed data model will have common attributes from both Debit Card Transactions and Credit Card Transactions real-time data feed. A feed application can follow one or more of these feed data models. like Credit Card Transactions data feed could follow Card Transaction and Location feed data model.

feed data models are configured in etc/models/feed_data_models/<model-name>/data_model.json.

below is the example of a solution containing four feed data model definition:

$tree etc/model/feed_data_models/
etc/model/feed_data_models/
├── identity
│   └── data_model.json
├── top_channels_prediction
│   └── data_model.json
├── topup
│   └── data_model.json
└── usage
    ├── data_model.json
    ├── enrichment_functions.json
    └── enrichment_scorers.json

The name of feedDataModels are Identity, Top Channel Prediction, Topup and Usage.

Example Feed data model:

{
    "attributes": [
        {
            "name": "MSISDN",
            "required": true,
            "type": "String"
        },
        {
            "name": "calledNumber",
            "required": false,
            "type": "String"
        },
        {
            "name": "callingNumber",
            "required": false,
            "type": "String"
        },
        {
            "name": "lastMSISDN",
            "required": false,
            "type": "String"
        },
        {
            "name": "ts",
            "required": true,
            "type": "Int64"
        }
    ],
    "name": "Identity",
    "profiles": [
        {
            "keys": [
                "MSISDN",
                "calledNumber",
                "callingNumber",
                "lastMSISDN"
            ],
            "name": "Customer"
        }
    ]
}

• attributes is a list of attributes of the model.
• name is the name of feed data model.
• profiles is a list of profile that can be realted to this feed model.
• profiles.name is the name of profile which can be related to this feed model for lookup or update purpose.
• profiles.keys is the list of attribute which can be used as keys to perform operation for the given profile. one of these keys should be not nullable.

Once feed data models are configured in feed_data_models directory, we need to point to it in product configuration (etc/drive.json)

The feed data models are configured in drive.feedDataModels section of product configuration etc/drive.json:

..
..
"drive": {
    "feedDataModels": [
          "Identity",
          "Top Channels Prediction",
          "Topup",
          "Usage"
      ],
}
..
..
"masterProfile": "Customer",
..
..