In high-dimensional data, which of the following becomes unreliable?

In high-dimensional data contexts, R-squared can become unreliable due to its sensitivity to the number of predictor variables included in a model. R-squared measures the proportion of variance in the dependent variable that can be explained by the independent variables in a regression model. With a high-dimensional dataset, where there are many predictors (potentially more than observations), R-squared tends to increase as more variables are added to the model, even if those additional variables do not provide meaningful insight or predictive power. This leads to an inflated perception of the model’s explanatory capability, a phenomenon known as overfitting.

In contrast, the other measures mentioned—such as the residual sum of squares, adjusted R-squared, and the standard error of the estimate—provide different perspectives or adjustments that can address or mitigate the issues that arise in high dimensions. For instance, adjusted R-squared specifically accounts for the number of predictors and adjusts the value of R-squared downwards based on the number of predictors relative to the number of observations. Thus, while R-squared itself can mislead in high-dimension scenarios, the adjusted R-squared works to provide a more realistic assessment, making it a more reliable measure in those cases.