A goal programming model for the interaction effects in multiple nonlinear regression
SURESH CHAND SHARMA1, DEVENDRA SINGH HADA2 and UMESH GUPTA3
1Department of Mathematics, University of Rajasthan, Jaipur, Rajasthan, India
Suresh chand [email protected]
2Department of Mathematics, Kautilya Institute of Technology and Engineering, ISI – 16, RIICO Institutional Complex, Sitapura, Jaipur,
Rajasthan, India [email protected]
3The Icfai Institute of Science and Technology, The Icfai University, Central Hope Town, Rajawala Road, Selaqui,
Dehradun, Uttarakhand, India [email protected]
ABSTRACT
Goal programming has proven a valuable mathematical programming form in a number of venues. Goal programming model serves a valuable purpose of cross checking answers from other methodologies. Likewise, multiple regression models can also be used to more accurately combine multiple criteria measures that can be used in GP model parameters. Those parameters can include the relative weighting and the goal constraint parameters. This paper is mainly focused on analyzing interaction effects in the context of multiple nonlinear regression equations. A comparison between the least square method and the linear goal programming method to solve the multiple nonlinear regression equations with two way interaction effects has been shown using MSExcel and TORA computer software packages.
Key words : Goal Programming; Multiple Regression;
Interaction Effects; Least Square Method.
1. INTRODUCTION
Linear regression is the oldest
and most widely used predictive model in biological, behavioral and social sciences to describe possible relation-
ships between variables. Linear regression finds application in a wide range of environmental science applications.
The earliest form of regression was the method of least squares, which was published by Legendre2 and by Gauss6. Multiple linear regression (MLR) analysis is a statistical tool for understanding the relationship between two or more variables. The main idea of a MLR analysis is to underst and t he relat ions hip between several independent variables and a single dependent variable. A number of methods for the estimation of the regression parameters are available in the literature. These include methods of minimizing the sum of absolute residuals, minimizing the maximum of absolute residuals and minimizing the sum of squares of residuals15, where the last method of minimizing the sum of squares of residuals popularly known as least square methods is commonly used.
Goal programming has proven a valuable tool in support of decision making. The first publication using GP was the form of a constrained regression model was used by Charnes et al.4. There have been many books devoted to this topic over past years (Ijiri11; Lee13; Spronk14; Ignizio10 and others).
This tool often represents a substantial improvem ent in the mode ling and analysis of multi-objective problems (Charnes and Cooper5; Eiselt et al.8; Ignizio9). By minimizing deviation the GP model can generate decision variable values that are the same as the beta
va lues in some types of m ult iple regression models.
An interaction occurs when the magnitude of the effect of one inde- pe ndent va riable on a de pe ndent variable varies as a function of a second independent variable. This is also known as a moderation effect, although some have more strict criteria for moderation effects than for interactions.
Now days interaction effects through regression models is a widely interested area of investigation as there has been a great deal of confusion about the analysis of moderated relationships involving continuous variables. Alken and West1 and Jaccard et al.12 have analyzed such interaction effects, further this method was applied into several models by the researchers, for example, Curran7 applied into hierarchical linear growth models.
2. Two way interaction effects in Multiple Regression
The regression equation used to analyze and int erpret a 2-way interaction is:
yir=b0+b1Xi+b2Zi+b3Xi2
+b4Zi2
+b5XiZi+ei, i = 1, 2,…, m.
Where, b0, b1 and b2 are the para- meters to be estimated and e is the error components which are assumed to be normally and independently distributed with zero mean and constant variance.
The linear absolute residual method requires us to estimate the values of these unknown parameters so as to
minimize i=m1|yi-yir|.
3. Linear goal programming formulation Let yi be the ith goal, di+ be positive deviation from the ith goal and di- be the negative deviation from the ith goal. Then the problem of minimizing
i=
m
1|yi-yir| may be reformulated as Minimize i=m1(di++di-)
Subject to:
a0+a1Xi1+a2Xi2+a3Xi3+a4Xi4+a5Xi5+di+-di-
=yiG , di+ 0 di- 0
and a0,a1,a2,a3,a4,a5, are unrestricted.
i= 1, 2, …, m.
Where Xi2, Zi2 and Xi Zi are taken as Xi3, Xi4 and Xi5 respectively to formulate the multiple nonlinear regression problem into linear goal programming model.
4. Relationship between the least square method and the linear goal programming method
Relationship between two methods can be established by taking a simple example. We consider a regression equation of Y on X and Z. The data for illustration is:
y x z
7.88 3 2
7.43 2 1
8.38 4 3
7.42 2 1
7.97 3 2
7.49 2 2
8.84 5 3
8.29 4 2
T he s olut ion of t he a bove re gres sion problem t hrough lea st square method using MSExcel gives:
b0=6.9215, b1 = 0.0001, b2 = 0.3181, b3=0.0602, b4= -0.0557, b5= 0.0001 and hence the corresponding estimated regression equation is
yir =6.9215+0.0001Xi + 0.3181Zi + 0.0602Xi2 -0.0557Zi2 + 0.0001Xi Zi
Where i= 1, 2, …, 8.
and Minimize i=18 |yi-yir|=0.2826.
Reformulating the above problem into linear goal programming model: Minimize
i=18
(di++di-) Subject to:
a0+3a1+2a2+9a3+4a4+6a5+d1+-d1=7.88 a0+2a1+a2+4a3+a4+2a5+d2+
-d2
=7.43 a0+4a1+3a2+16a3+9a4+12a5+d3+-d3=8.38 a0+2a1+a2+4a3+a4+2a5+d4+-d4=7.42 a0+3a1+2a2+9a3+4a4+6a5+d5+-d5=7.97
a0+2a1+2a2+4a3+4a4+4a5+d6+-d6=7.49 a0+5a1+3a2+25a3+9a4+15a5+d7+-d7=8.84 a0+4a1+2a2+16a3+4a4+8a5+d8+-d8=8.29 di+ 0, i = 1, 2, ..., 8
di 0, i = 1, 2, ..., 8
ai are unrestricted, i = 0, 1, 2, …,5.
The solution of the above formulated problem through simplex method using TORA computer software package:
a0 = 6.74, a1 = 0.28, a2 = 0.045, a3 = 0.01, a4 = -0.015, a5 = 0.03 Thus the estimated regression equation via the goal programming method is yiG=6 . 74+ 0. 2 8 Xi1+0 . 04 5 Xi2+0 . 0 1 Xi3 0.015Xi4+0.03Xi5, i = 1, 2, …, 8
Or
yi G= 6 . 7 4 + 0 . 2 8 Xi+ 0 . 0 4 5 Zi+ 0 . 0 1 Xi2 0.015Zi2+0.03XiZi, i = 1, 2, …, 8
and
Minimize i=18 |yi-yiG|=0.1.
T hus,Minimize i=18 |yi-yiG|<Minimize
i=1
8 |yi-yir|.
5. CONCLUSION
S olving t he a bove m ultiple nonlinear regression equation with two way interaction effect via least square
method and linear goal programming model clearly shows that Minimize
i=
m
1|yi-yiG|<Minimizei=m1|yi-yir|, where yiG be the estimate of the th response using goal programming technique and yir be the estimate using the least square method.
Hence, it is concluded that the goal programming technique provide better estimate of the multiple nonlinear regression parameters with two way interaction effect than the least square method.
6. REFERENCES
1. Alken, L.S. and West, S.G., Multiple Regression: Testing and Interpreting Interactions, Thousand Oaks: Sage Publications (1991).
2. A.M. Legendre, Nouvelles méthodes pour la détermination des orbites des cométes. “Sur la Méthode des moindres quarrés” appears as an appendix (1805).
3. Charnes, A., Cooper, W. W. and Ferguson, R., Optimal Estimation of Executive Compensation by Linear Programming, Management Science, Vol. 1, No. 2, pp 138 – 151 (1955).
4. Charnes, A. and Cooper, W. W., Goal Programming and Multiple Objective Optimizations, Eur. J.
Operat. Res., Vol. 1, 39–54 (1977).
5. C.F. Gauss, Theoria Motus Corpo- rum Coelestium in Sectionibus Conicis Solem Ambientum (1809).
6. Curran, P. J., Bauer, D. J. and
Willoughby, M. T., Testing Main Effects and Interactions in Hiera- rchical Linear Growth Models, Psychological Methods, Vol. 9, No.
2, 220-237 (2004).
7. Eiselt, H. A., Pederzoli, G. and Sandblom, C.L., Continuous Opti- mization Models, W De G, New York (1987).
8. Ignizio, J. P., A Review of Goal Programming – A Tool for Multio- bjective Analysis, J. Opl. Res. Soc., Vol. 29, No. 11, 1109–1119 (1978).
9. Ignizio, J.P., Introduction to Linear Goal Programming, thousand Oaks, CA: sage Publications (1986).
10. Ijiri, Y., Management Goals and Accounting for Control, Amsterdam:
North-Holland Publishing Company (1965).
11. Jaccard, J., Turrisi, R. and Wan, C. K. (1990).
12. Lee, S. M., Goal Programming for Decision Analysis, Philadelphia:
Auerbach Publishers Inc. (1972).
13. Spronk, J., Interactive Multiple Goal Programming: Application to Financial Planning, Amsterdam:
Martinus Nijhoff (1981).
14. Weisberg, S., Applied linear regre- ssion, 2nd edition, John Wiley and Sons, inc. New York (1985).
