DPTK Scheduling Jobs

DPTK provides background tasks for automated processing of user pseudonymization and automated cleanup of DPTK reports.

For configuration parameters please refer DPTK configuration

Following section provides information about how DPTK can be monitored to ensure the jobs are running correctly or to identify potential issues that were encountered during processing

All DPTK actions and states log information that can be leveraged to understand what it happening, and whether something didn’t process as expected

To check log please follow below steps:

  • Goto websphere console
  • Select Troubleshooting > Logging and tracing > DPTKCluster_server1 > Diagnostic trace service > Change log detail levels
  • Add com.ibm.ess.ic.dptk.scheduling.jobs package in runtime as shown in below screenshot
  • scheduling-job-logs

Note: This trace setting will generate logs for both Profile pseudonymization and Report Deletion jobs. And you can find those traces in the trace.log file.

Logs For Profile Pseudonymization:

  • Scheduling job is initialized successfully
com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob init created timer:{timer})

com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob init First timeout: {first timeout} - schedule: {corn job expression}
  • Scheduling Job is failed to initialize
com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob init Failed to initialize Job '{JOB_NAME}

Reason: The EJB timer for the mentioned job seems to have failed initializing. This could be related to the configured schedule this job should run at not being valid, the server's EJB Timer service settings not being properly configured or the underlying database communication for persisting EJB information failing. 

  • Scheduling Job is disabled
com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob anonymizeProfilesScheduledJob < execute done! - sync job is disabled - see property '{}'
  • Job is already running
com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob performAnonymization Anonymization is already started or still running!
  • If user is active
com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob performAnonymization User '{user guid}' is active, there is no need to evaluate the pseudonymization.
  • User is already pseudonymized
com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob shouldPseudonymize < false - user already pseudonymized
  • User excluded from pseudonymization
com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob shouldPseudonymize < false - user is excluded from pseudonymization
  • User is inactive
com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob performAnonymization User '{}' is inactive and should be pseudonymized.
  • Failed to retrieve report
com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob performAnonymization Failed retrieving report with id '{}'. Report: {}

Reason: This could be we are not getting report from dptk reports store i.e. it is null . 

  • Failed to anonymizing user
com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob performAnonymization Failed anonymizing user '{}' -> {}

Reason: This could be error while removing content item during anonymization Or Deletion of User Related Data from connected apps (e.g. ICXT, Expertise, Crowdsourcing ) etc . 


com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob performAnonymization Pseudonymization of profile entry attributes failed - adjustment of information for report failed too:

Reason : This is Exception  while adding profile entry attributes to Pseudonymization User into the report
  • Profile Successfully anonymized, but persisting report failed
com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob performAnonymization Profile was successfully anonymized, but persisting report '{}' failed

Reason : This could be   error while storing  updated Pseudonymized report to dptk report store. 

  • Profile Successfully Anonymized
com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob performAnonymization Profile of user '{}' was successfully anonymized. Finalize report...
  • Job Execution is completed
com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob anonymizeProfilesScheduledJob < execute done! - {} of {} users were stored in the DptkUserstateStore ({} failed, {} deletions
  • Job Execution failed
com.ibm.ess.ic.dptk.scheduling.jobs.AnonymizeSchedulingJob anonymizeProfilesScheduledJob < FAILED

Reason : This could be reason we are not able to perform Pseudonymization of profile in report

Logs for Report Deletion:

  • Scheduling job is initialized successfully
com.ibm.ess.ic.dptk.scheduling.jobs.DeletionSchedulingJob init created timer:{timer})

com.ibm.ess.ic.dptk.scheduling.jobs.DeletionSchedulingJob init First timeout: {first timeout} - schedule: {corn job expression}
  • Scheduling Job is failed to initialize
com.ibm.ess.ic.dptk.scheduling.jobs.DeletionSchedulingJob init Failed to initialize Job '{JOB_NAME}


Reason: The EJB timer for the mentioned job seems to have failed initializing. This could be related to the configured schedule this job should run at not being valid, the server's EJB Timer service settings not being properly configured or the underlying database communication for persisting EJB information failing. 

  • If Report expired date is greater than current time (Job running time).
DeletionSched 1 com.ibm.ess.ic.dptk.scheduling.jobs.DeletionSchedulingJob deleteExpiredReportsScheduledJob [Skip deletion] - the report expired date it is not in the past - [id:{id}, category: {categoryid}, expired:{expired datetime}, now:{current datetime}]


  • If report is invalid.
com.ibm.ess.ic.dptk.scheduling.jobs.DeletionSchedulingJob deleteExpiredReportsScheduledJob Skipping report {reportid}- it doesn't seem to be valid
  • If report is incomplete
com.ibm.ess.ic.dptk.scheduling.jobs.DeletionSchedulingJob deleteExpiredReportsScheduledJob [Skip deletion] - exclude incomplete reports - [id: {reportId}, category: {Category}, completed: {report completion status}] ...


Reason: Report Seems to be incomplete so its deletion is skipped.

  • If Report is excluded from deletion
com.ibm.ess.ic.dptk.scheduling.jobs.DeletionSchedulingJob deleteExpiredReportsScheduledJob [Skip deletion] – ex0clude excluded from deletion flagged reports - [id: {reportId}, category: {category}, completed: {completion status}] ...
  • Report is successfully deleted
com.ibm.ess.ic.dptk.scheduling.jobs.DeletionSchedulingJob deleteExpiredReportsScheduledJob The report was removed - [id: {reportId}, category: {category}, expired: {expired datetime}] ...
  • Job Execution is completed
com.ibm.ess.ic.dptk.scheduling.jobs.DeletionSchedulingJob deleteExpiredReportsScheduledJob < execute done - start: {}, duration: {}ms, {} expired reports were removed

Note : DPTK administrators can view statistics for the background job at HOSTNAME/dptk/#/pseudonymizations via the “Show Statistics” button. statistics_1

statistics_2

Irregularities in the execution times and results or less entries than expected given the configuration could be indicators that the job is not functioning normally. The log indicators mentioned in section Pseudonymization Logs can be used to get a better understanding of the job’s status.