Random Forest Regression

Definition of Regression tree: A regression tree (a decision tree with a numerical target variable) is a machine learning algorithm where clusters of target output variables with corresponding input variables are determined following optimal classification criteria derived from the training data. This is the common definition for Regression tree, other definitions can be discussed in the article

Short introduction

A simple regression tree algorithm starts with defining an optimal splitting value separating input and corresponding output training data into two clusters (subdomains of input training data and corresponding output data). In the following step each cluster is split again, till at the end of each tree branch a cluster is left that contains only a small more or less homogeneous subdomain of the output data (a so-called leaf). The training of the regression tree on known output data is based on finding the most efficient split for each input variable at each level of the regression tree. This consists of determining for each split the average value of the target variables in the two clusters and searching the optimal split that minimizes the total sum of the squared residuals of the target training variables with respect to the average value in each of the two clusters. The simple regression tree tends to overfit the training data; the noise (observation inaccuracies, random fluctuations) in the training data is reflected in the regression tree. Noise can be minimized by considering a multitude of regression trees, each trained on a subset of the training data and a subset of the input variables. The subsets are chosen at random and some of the training data may appear more than once in the subsets. This multitude of regression trees is called a random forest. The final output is the mean of the outputs of all the regression trees. By constructing all possible different random forests, and comparing the final outputs with test data, the random forest with the best prediction will be selected as the final choice. Some regression trees make use of a boosting algorithm (Adaboost) to enhance the performance of a weak classifier by attributing different weights to the outcomes of the forest of random regression trees.

Analysis technique	Strengths	Limitations	Application example
Prediction tool based on machine learning from training data	* Handles nonlinear relationships * Does classification and regression * Resilient to data noise and data gaps * Computationally efficient * Low overfitting risk	* Black box, no easy interpretation of results, no probability estimates * Less efficient if many trees * Not reliable outside the range of trained situations transformation	Time series forecasting Pattern recognition from images, e.g. interpretation remote sensing images

For more detailed explanations see:

StatQuest: Regression Trees, Clearly Explained by Josh Starmer

StatQuest: Random Forests Part 1 - Building, Using and Evaluating by Josh Starmer

Wikipedia Machine learning decision tree

Wikipedia Random forest

Related articles

Data analysis techniques for the coastal zone

The main author of this article is Job Dronkers Please note that others may also have edited the contents of this article. Citation: Job Dronkers (2024): Random Forest Regression. Available from http://www.coastalwiki.org/wiki/Random_Forest_Regression [accessed on 22-11-2024] For other articles by this author see Category:Articles by Job Dronkers For an overview of contributions by this author see Special:Contributions/Dronkers J