Random Forest

What is Stacking of Models in Machine Learning?

by

The last Ensemble method we will discuss in this series is called stacking (short for stacked generalization). It is based on a simple idea: instead of using trivial functions (such as hard voting) to aggregate the predictions of all predictors in an ensemble, why don’t we train a model to perform this aggregation? Figure below … Read more

Titanic – Advanced Feature Engineering Tutorial

by

Introduction I decided to write this kernel because Titanic: Machine Learning from Disaster is one of my favorite competitions on Kaggle. This is a beginner level kernel which focuses on Exploratory Data Analysis and Feature Engineering. A lot of people start Kaggle with this competition and they get lost in extremely long tutorial kernels. This is a short kernel compared … Read more

Gradient Boosting in Machine Learning

by

Another very popular Boosting algorithm is Gradient Boosting. Just like AdaBoost,Gradient Boosting works by sequentially adding predictors to an ensemble, each one correcting its predecessor. However, instead of tweaking the instance weights at every iteration like AdaBoost does, this method tries to fit the new predictor to the residual errors made by the previous predictor. … Read more

Random Forests | Machine Learning from Scratch

by

As we have discussed, a Random Forest is an ensemble of Decision Trees, generally trained via the bagging method (or sometimes pasting), typically with max_samples set to the size of the training set. Instead of building a BaggingClassifier and passing it a DecisionTreeClassifier, you can instead use the RandomForestClassifier class, which is more convenient and … Read more

What is Bagging and Pasting? Machine Learning from Scratch

by

Introduction One way to get a diverse set of classifiers is to use very different training algorithms, as just discussed. Another approach is to use the same training algorithm for every predictor, but to train them on different random subsets of the training set. When sampling is performed with replacement, this method is called bagging … Read more

ROC and AUC in Evaluating Classification Models

by

In the dynamic world of business, where data-driven decisions reign supreme, the accuracy and reliability of classification models play a pivotal role. Whether you’re involved in lead scoring or any other binary classification system, understanding the intricacies of evaluation metrics is key. Among these, the ROC (Receiver Operating Characteristic) curve and its integral companion, AUC … Read more

Information Gain in Machine Learning

by
Information Gain

Information Gain (IG) is critical in machine learning and decision tree algorithms, particularly in data classification and 𝐟𝐞𝐚𝐭𝐮𝐫𝐞 𝐬𝐞𝐥𝐞𝐜𝐭𝐢𝐨𝐧. Information Gain Information Gain is a concept used in the field of machine learning and decision trees to measure the effectiveness of an attribute in classifying a dataset. It is commonly employed in the construction of … Read more