Decision Tree | Sample Question

Question

$True$ or $False?$ If decision trees such as the ones we built in class are allowed to have decision nodes based on questions that can have many possible answers (e.g. “What country are you from) in addition to binary questions, they will in general tend to add the multiple answer questions to the tree before adding the binary questions

Answer 1

The statement is generally FALSE. The order in which decision trees add nodes, whether they are based on multiple answers or binary questions, depends on the specific algorithm and the optimization criteria used during the construction of the tree.

Traditional decision tree algorithms like ID3 (Iterative Dichotomiser 3) and C4.5 usually work with binary splits. They choose the attribute that provides the best binary split at each node based on a certain criterion (e.g., information gain or Gini impurity). These algorithms may not naturally handle questions with multiple answers in their standard form.

However, there are variations and extensions of decision tree algorithms that can handle multiway splits or questions with multiple answers. These modifications might be designed to handle categorical variables with more than two categories. Examples include C5.0 and Random Forests, which can incorporate multiway splits.

In any case, the specific behavior depends on the implementation details of the decision tree algorithm being used. The idea that decision trees will always tend to add multiple answer questions before binary questions is not universally true and depends on the criteria and considerations used during the construction of the tree.