How to Train XGBoost with Imbalanced Data Using Scale_pos_weight

Python

In this example we train a binary classification model using XGBoost with data that is imbalanced.

To remedy this we calculate spw which is the number of records with a negative target (records that have value 0 in y_train) divided by the number of positive records (those with a value 1 in y_train)

Finally this weight is passed to the model as the scale_pos_weight parameter.

 1|  from xgboost import XGBClassifier
 2|  
 3|  # Step 1: Calculate the ratio negative to positive 
 4|  # records in y_train
 5|  positive_records = y_train.sum()
 6|  negative_records = len(y_train) - positive_records
 7|  spw = negative_records / positive_records
 8|  
 9|  # Step 2: Pass this ratio to the scale_pos_weight 
10|  # parameter
11|  model = XGBClassifier(booster='gbtree', 
12|  			objective='binary:logistic', 
13|  			max_depth=12, learning_rate=0.1, 
14|  			n_estimators=10, 
15|  			scale_pos_weight=spw, 
16|  			random_state=101, 
17|  			n_jobs=-1)
18|  
19|  # Step 3: Fit model
20|  model.fit(X_train, y_train)
analyseup
XGBoost | Classification | Scale_pos_weight | Imbalanced
Did you find this snippet useful?

Sign up for free to to add this to your code library

Remove Stop Words from Text in DataFrame Column
Python

Pandas | Nltk | Nlp

152
LightGBM Custom Loss Function
Python

LightGBM | Loss function

134
Tuning XGBoost Hyperparameters with Grid Search
Python

XGBoost | Tuning

129
How to Convert DataFrame Values Into Percentages
Python

Lambda | Percentages | Pipe

122
How to Scale Data Using Standard Scaler But Keep Column Names
Python

Scaler | Standard

94
LightGBM Hyperparameter Tuning with GridSearch
Python

Python | LightGBM | Hyperparameter tuning | Gridsearch

93
Calculating Root Mean Squared Error (RMSE) with Sklearn and Python
Python

Rmse

89
How to Train a Catboost Classifier with GridSearch Hyperparameter Tuning
Python

Catboost | Hyperparameter tuning

70
Dynamically Create Columns in Pandas Dataframe
Python

Pandas | Columns

70
Overwriting a Python Logging File Instead of Appending
Python

Logging

67