Sklearn Bagging Classifier - Training a Decision Tree Classification Model with Bagging
Python
This code snippet demonstrates how to train a bagging classification model using the Sklearn BaggingClassifier. In this example we use a Decision Tree Classifier as the base estimator but any other classification model can be used.
The base estimator is initialised first with all hyperparameters declared and then this is passed as a hyperparameter to the BaggingClassifier.
1| from sklearn.tree import DecisionTreeClassifier 2| from sklearn.ensemble import BaggingClassifier 3| from sklearn.metrics import classification_report, log_loss, roc_auc_score 4| 5| # Step 1: Initialise Decision Tree Classifier 6| decision_tree = DecisionTreeClassifier(criterion='gini', 7| max_depth=None, 8| min_samples_split=5, 9| min_samples_leaf=5, 10| random_state=101) 11| 12| # Step 2: Initialise Bagging Classifier and set hyperparameters 13| model = BaggingClassifier(base_estimator=decision_tree, 14| n_estimators=5, 15| max_samples=0.8, 16| max_features=0.8, 17| bootstrap=True, 18| bootstrap_features=False, 19| random_state=1) 20| 21| # Step 3: Fit model 22| model.fit(X_train, y_train) 23| 24| # Step 4: Make predictions for test data & evaluate performance 25| y_pred = model.predict(X_test) 26| print('Classification Report:',classification_report(y_test, y_pred)) 27| print('Log Loss:',log_loss(y_test, y_pred)) 28| print('ROC AUC:',roc_auc_score(y_test, y_pred))
133
121
116
109