LightGBM Hyperparameter Tuning with GridSearch
Python
This code snippet performs hyperparameter tuning for a LGBMRegressor model using Grid Search with 3-fold cross validation. It defines a parameter grid with hyperparameters, initializes the LGBMRegressor estimator, fits the model with the training data, and prints the best parameters found by the Grid Search. It also plots the feature importances using the best model, makes predictions for the test data, and evaluates the performance of the model using various metrics such as RMSE, MAE, max error, and explained variance score.
1| import lightgbm as lgb 2| from sklearn.metrics import mean_squared_error, mean_absolute_error, max_error, explained_variance_score, mean_absolute_percentage_error 3| import matplotlib.pyplot as plt 4| from sklearn.model_selection import GridSearchCV 5| 6| # Step 1: Define parameter grid for hyperparameter tuning 7| param_grid = { 8| 'learning_rate': [0.1, 0.01], 9| 'n_estimators': [50, 100, 150], 10| 'max_depth': [4, 6], 11| 'colsample_bytree': [0.7, 0.8, 0.9], 12| 'subsample': [0.7, 0.8, 0.9], 13| 'min_child_samples': [1, 5, 10] 14| } 15| 16| # Step 2: Initialize LGBMRegressor estimattor 17| estimator = lgb.LGBMRegressor(objective='regression', 18| random_state=101) 19| 20| # Step 3: Initalise Grid Search with 3-fold cross validation and fit model 21| model = GridSearchCV(estimator=estimator, 22| param_grid=param_grid, 23| cv=3, 24| n_jobs=-1, 25| scoring='neg_root_mean_squared_error') 26| model.fit(X_train, y_train) 27| 28| # Step 4: Print best parameters 29| best_params = model.best_estimator_ 30| print(best_params) 31| 32| # Step 5: Plot feature importances 33| features = X_train.columns 34| importance_values = best_model.feature_importances_ 35| 36| plt.barh(y=range(len(features)), 37| width=importance_values, 38| tick_label=features) 39| plt.show() 40| 41| # Step 6: Make prediction for test data & evaluate performance 42| y_pred = best_model.predict(X_test) 43| print('RMSE:',mean_squared_error(y_test, y_pred, squared=False)) 44| print('MAE:',mean_absolute_error(y_test, y_pred)) 45| print('Max Error:',max_error(y_test, y_pred)) 46| print('Explained Variance Score:',explained_variance_score(y_test, y_pred))
149
132
127
119