Entropy

Gini Impurity or Entropy? How to decide the root node in decision tree?

by

By default, the Gini impurity measure is used, but you can select the entropy impurity measure instead by setting the criterion hyperparameter to “entropy”. The concept of entropy originated in thermodynamics as a measure of molecular disorder: entropy approaches zero when molecules are still and well ordered. It later spread to a wide variety of … Read more

Decision Trees | Machine Learning from Scratch

by

Like SVMs, Decision Trees are versatile Machine Learning algorithms that can perform both classification and regression tasks, and even multioutput tasks. They are very powerful algorithms, capable of fitting complex datasets. For example, you trained a DecisionTreeRegressor model on the California housing dataset, fitting it perfectly (actually overfitting it).Decision Trees are also the fundamental components … Read more