Fairness Metrics

The fairness metrics can identify data bias. Three metrics—Statistical Parity Difference, Disparate Impact, and Theil Index—check for unwanted bias in the dataset. Both measure discrimination (i.e., deviation from fairness).

1. Statistical Parity measures the difference between the majority and protected classes receiving a favorable outcome. To be fair, this measure must be equal to 0.
2. Disparate impact compares the proportion of individuals who receive a favorable outcome for two groups: a majority group and a minority group. To be fair, this measure must be equal to 1.
3. Theil Index measures inequality among individuals in the group for a classification problem type. Ideally, it should be zero for perfect fairness, and bias increases with the increase in value greater than zero.