toad.metrics module

toad.metrics.KS(score, target)[source]

calculate ks value

Parameters:
  • score (array-like) – list of score or probability that the model predict
  • target (array-like) – list of real target
Returns:

the max KS value
Return type:

float
toad.metrics.KS_bucket(score, target, bucket=10, method='quantile', return_splits=False, **kwargs)[source]

calculate ks value by bucket

Parameters:
  • score (array-like) – list of score or probability that the model predict
  • target (array-like) – list of real target
  • bucket (int) – n groups that will bin into
  • method (str) – method to bin score. quantile (default), step
  • return_splits (bool) – if need to return splits of bucket
Returns:

DataFrame
toad.metrics.KS_by_col(df, by='feature', score='score', target='target')[source]
toad.metrics.SSE(y_pred, y)[source]

sum of squares due to error

toad.metrics.MSE(y_pred, y)[source]

mean of squares due to error

toad.metrics.AIC(y_pred, y, k, llf=None)[source]

Akaike Information Criterion

Parameters:
  • y_pred (array-like) –
  • y (array-like) –
  • k (int) – number of featuers
  • llf (float) – result of log-likelihood function
toad.metrics.BIC(y_pred, y, k, llf=None)[source]

Bayesian Information Criterion

Parameters:
  • y_pred (array-like) –
  • y (array-like) –
  • k (int) – number of featuers
  • llf (float) – result of log-likelihood function
toad.metrics.F1(score, target, split='best', return_split=False)[source]

calculate f1 value

Parameters:
  • score (array-like) –
  • target (array-like) –
Returns:

best f1 score float: best spliter
Return type:

float
toad.metrics.AUC(score, target, return_curve=False)[source]

AUC Score

Parameters:
  • score (array-like) – list of score or probability that the model predict
  • target (array-like) – list of real target
  • return_curve (bool) – if need return curve data for ROC plot
Returns:

auc score
Return type:

float
toad.metrics.PSI(test, base, combiner=None, return_frame=False)[source]

calculate PSI

Parameters:
  • test (array-like) – data to test PSI
  • base (array-like) – base data for calculate PSI
  • combiner (Combiner|list|dict) – combiner to combine data
  • return_frame (bool) – if need to return frame of proportion
Returns:

float|Series
toad.metrics.matrix(y_pred, y, splits=None)[source]

confusion matrix of target

Parameters:
  • y_pred (array-like) –
  • y (array-like) –
  • splits (float|list) – split points of y_pred
Returns:

confusion matrix witch true labels in rows and predicted labels in columns
Return type:

DataFrame