What is a major characteristic of decision trees?

A major characteristic of decision trees is that they are sensitive to noise and overfitting. Decision trees create a model by splitting the data into subsets based on feature values, which can lead to highly intricate rules that may capture the noise in the training data rather than the true underlying relationships. This sensitivity is particularly pronounced when the trees become very deep, as they can adapt too closely to the training data, resulting in poor generalization to unseen data.

The complexity of decision trees allows them to model intricate patterns, but it also means they can easily fit noise, making them vulnerable to overfitting. Overfitting occurs when a model becomes too tailored to the training dataset, missing out on broader trends that may be more applicable to future data.

In contrast, decision trees do not produce linear predictions nor do they inherently perform better with categorical data compared to quantitative data; they can handle both types effectively. While decision trees generally have some robustness against outliers due to their structure, they are still not immune to their influence, especially when the outliers significantly affect the decision rules. Therefore, understanding this sensitivity to noise and the risk of overfitting is crucial when utilizing decision trees in predictive modeling.