Prediction of Heteroscedastic Data Using Linear Regression and Various Machine Learning Models

Muhammad Danish Munir¹

1. College of Statistical Sciences, University of The Punjab, Lahore, Pakistan.

Section:Research Paper, Product Type: Journal-Paper
Vol.10 , Issue.1 , pp.14-19, Feb-2023

Online published on Feb 28, 2023

Copyright © Muhammad Danish Munir . This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
 

View this paper at   Google Scholar | DPI Digital Library

XML View     PDF Download

How to Cite this Paper

227 Views    317 Downloads    48 Downloads

Abstract :
According to OLS assumption, “variance of error term should be constant”. Whenever this assumption is violated then heteroscedasticity problem arises. If we apply the OLS method to heteroscedastic data then predicted values are inaccurate and our inference may be wrong about data. In this research, an attempt has been made to implement various machine learning techniques to predict the values of heteroscedastic data. We take two different types of heteroscedastic datasets and split each dataset into two sections, training dataset and a testing dataset with 80% and 20% ratios. We apply different machine learning models including Decision Tree, Random Forest, Gradient Boosting, K Nearest Neighbor, and also apply Linear regression on training datasets and predict the values of testing datasets. All machine learning models provide minimum values of the sum of the square as compared to Linear regression. Some of machine learning models shows good results, its predicted values are nearest to observed values. As the results machine learning models provides high r-square and a low sum of the square. This research shows that we can deal heteroscedastic data with machine learning models.

Key-Words / Index Term :
Machine Learning Prediction, Prediction of Heteroscedastic Data, Heteroscedasticity, Heteroscedastic Data

References :
[1] O. L. O. Astivia and B. D. Zumbo, “Heteroskedasticity in Multiple Regression Analysis?: What it is , How to Detect it and How to Solve it with Applications in R and Heteroskedasticity in Multiple Regression Analysis?: What it is , How to Detect it and How to Solve it with Applications in R,” Pract. Assessment, Res. Eval. Vol., vol. 24, 2019.
[2] L. Wang and X.-H. Zhou, “Assessing the Adequacy of Variance Function in Heteroscedastic Regression Models,” Biometrics, vol. 63, no. 4, pp. 1218–1225, 2007, [Online]. Available: http://www.jstor.org/stable/4541477
[3] J. S. Long and L. H. Ervin, “Using Heteroscedasticity Consistent Standard Errors in the Linear Regression Model,” Am. Stat., vol. 54, no. 3, pp. 217–224, Aug. 2000, doi: 10.1080/00031305.2000.10474549.
[4] Z. L. Yang and Y. K. Tse, “Tests of Functional Form and Heteroscedasticity,” Res. Collect. Sch. Econ., pp. 1–24, 2003, [Online]. Available: https://ink.library.smu.edu.sg/soe_research/775
[5] H. Midi, M. S. Rana, and A. H. M. R. Imon, “The performance of robust weighted least squares in the presence of outliers and heteroscedastic errors,” WSEAS Trans. Math., vol. 8, no. 7, pp. 351–361, 2009.
[6] Rahul, A. Gupta, A. Bansal, and K. Roy, “Solar energy prediction using decision tree regressor,” Proc. - 5th Int. Conf. Intell. Comput. Control Syst. ICICCS 2021, no. Iciccs, pp. 489–495, 2021, doi: 10.1109/ICICCS51141.2021.9432322.
[7] D. Karir, A. Ray, A. Kumar Bharati, U. Chaturvedi, R. Rai, and M. Khandelwal, “Stability prediction of a natural and man-made slope using various machine learning algorithms,” Transp. Geotech., vol. 34, no. September 2021, 2022, doi: 10.1016/j.trgeo.2022.100745.
[8] C. Bienefeld, E. Kirchner, A. Vogt, and M. Kacmar, “On the Importance of Temporal Information for Remaining Useful Life Prediction of Rolling Bearings Using a Random Forest Regressor,” Lubricants, vol. 10, no. 3, p. 48, 2022, doi: 10.3390/lubricants10030048.
[9] W. Cai, R. Wei, L. Xu, and X. Ding, “A method for modelling greenhouse temperature using gradient boost decision tree,” Inf. Process. Agric., vol. 9, no. 3, pp. 343–354, 2021, doi: 10.1016/j.inpa.2021.08.004.
[10] A. G. Klein, C. Gerhard, R. D. Büchner, S. Diestel, and K. Schermelleh-Engel, “The detection of heteroscedasticity in regression models for psychological data model fit of nonlinear structural equation models view project,” Psychol. Test Assess. Model., vol. 58, no. 4, pp. 542–568, 2016, [Online]. Available: https://www.researchgate.net/publication/311518028