Performance and fairness
Performance
Performance compares the reference window with the current one, using only rows that carried a label.
Each card shows the current value, the change against the reference and the Reference value.
Under the cards, a tolerance line explains the result, for example:
- F1 +0.443 against the reference; degraded when it falls by more than 0.05. 2,400 labelled reference rows, 604 labelled current rows.
- RMSE -222.201 (-1.4%) against the reference; degraded when it rises by more than 5%.
So for a classifier the tolerance is how far F1 may fall; for a regression it is how far RMSE may rise, relative to the reference. Past it, the model is Degraded and a finding opens. With no labelled rows, performance shows No labels. Set the tolerance in Monitoring settings.
Fairness
Fairness looks at each protected attribute you registered, group by group, on the current window.
Classifiers
Figure 1. A classifier: performance against the reference, and fairness for each protected attribute.
For each attribute (for example gender or age_band), a table lists every group with its Rows, Selection rate, True positive rate and False positive rate. Next to it:
Two flags can appear on an attribute:
- Selection-rate ratio: the ratio fell below Minimum selection-rate ratio (default 0.8, the four-fifths rule).
- Equalized odds: the difference rose above Maximum equalized odds difference (default 0.1).
Regressions
Figure 2. A regression: MAE, RMSE and R², with mean prediction and error gaps by group.
For each attribute, a table lists every group with its Rows, Mean prediction and MAE, next to the Mean prediction gap and Error gap between groups. The Error flag appears when groups differ too much. The Maximum equalized odds difference setting is also the regression tolerance between groups.
A flagged attribute marks the model Fairness: Flagged and opens a finding. Set both limits in Monitoring settings.