 DETRO

LightGBM Custom Loss Function

Python
Supervised Learning

LightGBM gives you the option to create your own custom loss functions. The loss function you create needs to take two parameters: the prediction made by your lightGBM model and the training data.

Inside the loss function we can extract the true value of our target by using the get_label() method from the training dataset we pass to the model. Using this and the prediction made by the model we can calculate the error between the actual target and the model's prediction which will be used to calculate the loss.

A custom loss function requires two values to be returned known as grad and hess. Grad is the first derivative of whatever loss function defined and hess is the second derivative.

In the example below we've used the custom loss function to create a simple mean squared error function to demonstrate this. The loss function is simply the square of the error, therefore the first derivative (Grad) is simply the error multiplied by 2 and the second derivative (Hess) is just 2.

Note: Hess and grad both need to be 1-dimensional arrays of the same length as the number of examples in our training data set. That's why in the example below, instead of just declaring hess = 2, we multiply the error array by 0 and then add 2 to ensure we have a 1-dimensional array that has the same length as our training data set with every element in the array set to 2.

import LightGBM as lgb

def custom_loss(y_pred, data):
y_true = data.get_label()
error = y_pred-y_true

#1st derivative of loss function

#2nd derivative of loss function
hess = 0 * error + 2

params = {"learning_rate" : 0.1}

training_data = lgb.Dataset(X_train , label = y_train)

model = lgb.train(train_set=training_data,
params=params,
fobj=custom_loss)

By detro - Last Updated Dec. 6, 2021, 11:53 p.m. Did you find this snippet useful?

RELATED SNIPPETS
4
Train Catboost Classifier with GridSearch
Python
Supervised Learning
3
Training a Model Using Stratified K-Fold
Python
Supervised Learning
3
Scoring Models with Sklearn MSE and MAE
Python
Supervised Learning
1
Logistic Regression with Sklearn
Python
Supervised Learning
1
Linear Regression with Sklearn
Python
Supervised Learning
1
Tuning XGBoost with Grid Search
Python
Supervised Learning
1
Top Contributors 82 75 36
Find Snippets by Language
Find Snippets by Use