toad.metrics module

toad.metrics.KS(score, target)[source]

calculate ks value

Parameters

  • score (array-like) – list of score or probability that the model predict

  • target (array-like) – list of real target

Returns

the max KS value

Return type

float

toad.metrics.KS_bucket(score, target, bucket=10, method='quantile', return_splits=False, **kwargs)[source]

calculate ks value by bucket

Parameters

  • score (array-like) – list of score or probability that the model predict

  • target (array-like) – list of real target

  • bucket (int) – n groups that will bin into

  • method (str) – method to bin score. quantile (default), step

  • return_splits (bool) – if need to return splits of bucket

Returns

DataFrame

toad.metrics.KS_by_col(df, by='feature', score='score', target='target')[source]
toad.metrics.SSE(y_pred, y)[source]

sum of squares due to error

toad.metrics.MSE(y_pred, y)[source]

mean of squares due to error

toad.metrics.AIC(y_pred, y, k, llf=None)[source]

Akaike Information Criterion

Parameters

  • y_pred (array-like) –

  • y (array-like) –

  • k (int) – number of featuers

  • llf (float) – result of log-likelihood function

toad.metrics.BIC(y_pred, y, k, llf=None)[source]

Bayesian Information Criterion

Parameters

  • y_pred (array-like) –

  • y (array-like) –

  • k (int) – number of featuers

  • llf (float) – result of log-likelihood function

toad.metrics.F1(score, target, split='best', return_split=False)[source]

calculate f1 value

Parameters

  • score (array-like) –

  • target (array-like) –

Returns

best f1 score float: best spliter

Return type

float

toad.metrics.AUC(score, target, return_curve=False)[source]

AUC Score

Parameters

  • score (array-like) – list of score or probability that the model predict

  • target (array-like) – list of real target

  • return_curve (bool) – if need return curve data for ROC plot

Returns

auc score

Return type

float

toad.metrics.PSI(test, base, combiner=None, return_frame=False)[source]

calculate PSI

Parameters

  • test (array-like) – data to test PSI

  • base (array-like) – base data for calculate PSI

  • combiner (Combiner|list|dict) – combiner to combine data

  • return_frame (bool) – if need to return frame of proportion

Returns

float|Series

toad.metrics.matrix(y_pred, y, splits=None)[source]

confusion matrix of target

Parameters

  • y_pred (array-like) –

  • y (array-like) –

  • splits (float|list) – split points of y_pred

Returns

confusion matrix witch true labels in rows and predicted labels in columns

Return type

DataFrame