stouputils.data_science.metric_dictionnary module#

This module contains the MetricDictionnary class, which provides a dictionary of metric names.

This is often used to log metrics to MLflow and to display them in the console easily.

This class contains the following metrics:

  1. Main metrics:

    • Area Under the Curve (AUC)

    • Area Under the Precision-Recall Curve (AUPRC)

    • Area Under the NPV-Specificity Curve (NEGATIVE_AUPRC)

    • Specificity (True Negative Rate)

    • Recall/Sensitivity (True Positive Rate)

    • Precision (Positive Predictive Value)

    • Negative Predictive Value (NPV)

    • Accuracy

    • F1 Score

    • Precision-Recall Average

    • Precision-Recall Average for Negative Class

  2. Confusion matrix metrics:

    • True Negatives (TN)

    • False Positives (FP)

    • False Negatives (FN)

    • True Positives (TP)

    • False Positive Rate

    • False Negative Rate

    • False Discovery Rate

    • False Omission Rate

    • Critical Success Index (Threat Score)

  3. F-scores:

    • F-beta Score (where beta is configurable)

  4. Matthews correlation coefficient:

    • Matthews Correlation Coefficient (MCC)

  5. Optimal thresholds for binary classification:

    • Youden’s J statistic

    • Cost-based threshold

    • F1 Score threshold

    • F1 Score threshold for the negative class

  6. Average metrics across folds:

    • Mean value of any metric across k-fold cross validation

  7. Standard deviation metrics across folds:

    • Standard deviation of any metric across k-fold cross validation

This Page