BigFix Insights Change Log

This topic lists out the change log in BigFix Insights.

March 31, 2025 release - Insights site version 28

Added features:

  1. Batching support for initial ETL and incremental ETL
  2. Support ingestion from live BFEnterprise database(beta)
  3. Parallelism among different entities for initial ETL & incremental ETL(multiple datastream)
  4. Use OpenQuery as standard for ALL entities.
  5. Enhanced logging for import log.
  6. Node.js upgrade to node 22.
  7. Allow disable ciphers for insights broker via env variable INSIGHTS_BROKER_DISABLE_CIPHERS
  8. Better failure recovery for each ETL stage
    • ETL can partially succeed.
    • Error during post core step will not crash ETL.
    • Error during step will not crash ETL.
    • Error during clean up stage will not crash ETL.

Fixed various bugs and defects:

  1. Orphaned devices are now getting completely removed from insights db.(devices which are deleted in BFEnterprise by audit trail cleaner)
  2. Fixed overflow issue for datasourcefixlet, new migration added.
  3. Fix collation error when retrieving mismatched site ids
  4. Newly added external sites should be excluded by default
  5. Lots of other minor bug fixes.
January 2022 Release – Insights 10.0.8
  • Fixes various bugs and defects
    • Fixes issues with the insights_table_metrics table population where duplicate key was possible
    • Fixes issues with tracking sequences for new entities
    • Fixes issue with Custom Attributes that would overwrite existing entries with the same name
    • Fixes issue with etl_metrics - adding support for binary type columns
    • Fixes issue where datasource_fixlets was not accurately setting hidden for custom content
  • Added features
    • Site inclusion changes from excluded to included now cause a last_sequence reset for all entities associated with this datasource
    • Custom Attributes view adds a selected-only filter
    • Custom Attributes view adds validation for user input
    • Removes possible sensitive data from all future error output.
    • datasource_sites now includes operator sites
    • [schema change] device_dimensions added colums:
      • agent_type
    • [schema change] datasource_sites added columns:
      • opsite_datasource_user_id
    • [schema change] datasource_fixlets adds columns:
      • user_remote_id
      • creation_time
      • last_modification_time
    • [schema change] added new entities (tables):
      • datasource_baselines
      • datasource_baseline_action_settings
      • datasource_baseline_action_settings_user_groups
      • datasource_baseline_component_actions
      • datasource_baseline_component_action_successes
      • datasource_baseline_component_groups
      • datasource_baseline_components
      • datasource_baseline_fields
      • datasource_baseline_relevances
      • datasource_custom_site_readers
      • datasource_custom_site_writers
      • datasource_device_baselines
      • datasource_external_site_visibilities
      • datasource_fixlet_actions
      • datasource_content_comments
      • datasource_fixlet_relevances
      • datasource_role_group_assignments
      • datasource_role_site_assignments
      • datasource_role_user_assignments
      • datasource_roles
      • datasource_site_users
      • datasource_users
      • etl_entity_inclusions

To enable optional entities, insert datasource id and entity name into dbo.etl_entity_inclusion table. Datasource_id can be found in dbo.datasources table and each entity name in src/entity from insights-etl.

Ex: This following script will enable datasource_baselines:

--Replace ? with the actual datasource id 
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name) 
VALUES(?, 'DatasourceBaseline')

To enable all the optional entities for specific datasource, use the script below:

--Replace ? with the actual datasource id

INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceBaseline')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceBaselineActionSetting')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceBaselineActionSettingsUserGroup')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceBaselineComponent')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceBaselineComponentAction')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceBaselineComponentGroup')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceBaselineComponentActionSuccess')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceBaselineField')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceBaselineRelevance')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceCustomSiteReader')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceCustomSiteWriter')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceDeviceBaseline')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceExternalSiteVisibility')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceFixletAction')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceContentComment')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceFixletRelevance')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceRoleGroupAssignment')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceRoleSiteAssignment')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceRoleUserAssignment')
INSERT INTO dbo.etl_entity_inclusions (datasource_id, entity_name)
VALUES(?, 'DatasourceRoleUserAssignment')