In this paper, we proposed the **Stochastic** **Restricted** **Liu** **Estimator** (SRLE) for logistic regression model when the linear **stochastic** restriction was available. In the sense of MSEM, we got the necessary and sufficient condition or sufficient condition that SRLE was superior to MLE, LLE, SRMLE and SRLMLE and Veri- fy its superiority by using Monte Carlo simulation. How to reduce the new esti- mation’s bias is the focus of our next step which guaranteed mean square error does not increase.

13 Read more

In this paper we compare recently developed preliminary test **estimator** called Preliminary Test **Stochastic** **Restricted** **Liu** **Estimator** (PTSRLE) with Ordinary Least Square **Estimator** (OLSE) and Mixed **Estimator** (ME) in the Mean Square Error Matrix (MSEM) sense for the two cases in which the **stochastic** restrictions are correct and not correct. Finally a numerical example and a Monte Carlo simulation study are done to illustrate the theoretical findings.

10 Read more

Ridge type estimators are used to estimate regression parameters in a multiple linear regression model when multi- colinearity exists among predictor variables. When different estimators are available, preliminary test estimation proce- dure is adopted to select a suitable **estimator**. In this paper, two ridge estimators, the **Stochastic** **Restricted** **Liu** **Estimator** and **Liu** **Estimator** are combined to define a new preliminary test **estimator**, namely the Preliminary Test **Stochastic** Re- stricted **Liu** **Estimator** (PTSRLE). The **stochastic** properties of the proposed **estimator** are derived, and the performance of PTSRLE is compared with SRLE in the sense of mean square error matrix (MSEM) and scalar mean square error (SMSE) for the two cases in which the **stochastic** restrictions are correct and not correct. Moreover the SMSE of PTSRLE based on Wald (WA), Likelihood Ratio (LR) and Lagrangian Multiplier (LM) tests are derived, and the per- formance of PTSRLE is compared using WA, LR and LM tests as a function of the shrinkage parameter d with respect to the SMSE. Finally a numerical example is given to illustrate some of the theoretical findings.

10 Read more

3.1 Simulator, Species Group, Site Index Class, and Management Regime The simulator, deﬁnitions of species group and site index class, and management regimes for each species group are the same as in **Liu** et al. (2009). In brief, a spatially explicit forest estate model called OPTIONS from DR Systems Inc. is used to determine impacts of various forest management ac- tivities at the forest and stand levels. The simulation lengths are 200-years. The six broad species groups are natural pine stands, traditionally managed pine stands, intensively managed pine stands, upland and bottom- land hardwoods, and oak-pine forests, abbreviated as NSOF, PSOF, IMP, UWDS, BHWD, and OAKP re- spectively. The seven site index classes are extremely low, very low, low, medium, high, very high, and ex- tremely high, abbreviated as ELOW, VLOW, LOW, MED, HIGH, VHIG, EHIG respectively. A management regime is composed of combinations of individual silvi- cultural treatments, such as regeneration, fertilization, thinning, genetically improved stock, etc.

15 Read more

There are only a few papers considering the statistical inference of the SGD method. Chen et al. (2016) proposed a method called the batch-mean procedure. Although com- putationally efficient and theoretically sound, the batch-means procedure substantially un- derestimates the variance of the SGD **estimator** in finite-sample studies, because of the correlations between the batch means. Li et al. (2017) presented a new method for statis- tical inference in M-estimation problems, based on SGD estimators with a fixed step size. However, this method is limited to M-estimation and fixed step size. Su and Zhu (2018) proposed a new method called HiGrad, short for Hierarchical Incremental GRAdient De- scent, which estimates model parameters in an online fashion and provides a confidence interval for the true population value. This method is also computationally efficient and theoretically sound, but it is not applicable to vanilla SGD estimators.

21 Read more

In this paper, we introduce new **Stochastic** **Restricted** **Estimator** for SUR model, defined by **Stochastic** **Restricted** **Liu** Type SUR **estimator** (SRLTSE) . The proposed **estimator** has to deal with multicollinearity in SUR model if there is a degree of uncertainty in the parameters restriction. Moreover, the superiority of (SRLTSE) was derived with respect to mean squared error matrix (MSEM) criterion. Finally, a simulation study was conducted. This simulation used standard mean squares error (MSE) criterion to illustrate the advantage between **Stochastic** **Restricted** SUR **estimator** (SRSE), **Stochastic** **Restricted** Ridge SUR **estimator** (SRRSE), and **Stochastic** **Restricted** **Liu** Type SUR **estimator** (SRLTSE) at by several factors.

The classical regression approach (including the SAR model) is widely used to predict an average value of a dependent variable (for given values of determinants). Another very practically important issue is estimation of unit’s efficiency level. Efficiency is usually considered as a ratio of results (a dependent variable) and resources used (determinants). There are some methodologies developed to estimate unit’s efficiency; many of them take a relative nature of the efficiency indicator into account. Frontier-based methods consist in constructing of a hypothetical set of 100% efficient units (an efficiency frontier) and estimating of unit’s efficiency as a distance from this frontier. **Stochastic** frontier model utilises probabilistic approach to the efficiency frontier and can be formalised as [5, 10]:

19 Read more

Focusing more on the RBM structure, Taylor et al. [4] introduced the Conditional RBM (CRM) to exploit the temporal relationship between consecutive frames in time-series data. Two more interactions for RBM structure, i.e. visible-to-visible and visible- to-hidden feed forward connections from previous time steps are added to the standard RBM structure. These connections are effectively incorporated to the formulation of bias terms making the learning process similar to RBM. Moreover, it can inherit most important properties of standard RBM such as simple, exact inference and efficient approximate learning. Later, Hinton [5] refined the CRBM structure by implementing the multiplicative interactions in their proposed Gated Conditional **Restricted** Boltzmann Machines (GCRBM). In this model, three sets of units, i.e. input, output and hidden units are first defined. Then instead of simply incorporating them via bias terms as CRBM, a three-way interaction is introduced to let the input units directly influence the interactions between units. By this way, the input units will be able to gate the basic function for reconstructing the output. The energy function is defined as

(vi). The Generalized Maximum Entropy (GME) **estimator** of β : Golan et al. (1996) introduced the Generalized Maximum Entropy (GME) **estimator** to resolve the multicollinearity problem. This **estimator** requires a number of support values supplied subjectively and exogenously by the researcher. The estimates as well as their standard errors depend on those support values. In a real life situation it is too demanding on the researcher to supply appropriate support values, which limits the application of GME.

local influence method. They detected that cases 10, 4, 15, 16, and 1 (in this order) were most anomalous obser- vations. Researchers [17-22] also studied the same data to identify influential cases in modified ridge regression **estimator** and **Liu** **estimator** using global influence, local influence and Cook’s minor perturbation methods and they identified 16, 4, 1, 10 and 15 were the most influential cases but the order of magnitude is changed.

From a statistical perspective, the importance of those laws stem from the fact that the first one gives in an asy- mptotic sense the smallest 100% confidence interval for the parameter, while the second one gives an almost sure lower bound on the accuracy that the **estimator** can achi- eve.

compared with difference-based **estimator** ˆ diff by using MSE criterion. The properties of each of difference-based ridge **estimator** and **Liu** type **estimator** for the partially linear semiparametric model were studied when the errors are independent with equal variance and compared the two estimators through MSE and were extended the results to errors which have the problems of heterogeneity and autocorrelation(7). Also new estimates of shrinkage parameter in generalized difference-based ridge **estimator**(8) were proposed for semiparametric regression model, then the risk function of the **estimator** was calculated and the generalized difference -based **estimator** was introduced to the vector of parameters of semiparametric regression model when errors are correlated(9) and suggested the generalized **restricted** difference- based **Liu** **estimator** when there is a non **stochastic** constraint. A difference - based almost unbiased **Liu** **estimator** (DBAULE)(10), was proposed to estimate the linear part in a partial linear model, and studied its characteristics and the generalized difference-based ridge **estimator** was proposed to the vector of parameters in a partial linear model when the errors are dependent(11) and was compared the performance of proposed **estimator** with the generalized **restricted** difference-based ridge **estimator** by using MSE criterion. Also a Jackknifed difference- based ridge **estimator** (12) was proposed in partial model; the proposed estimate was compared with difference- based ridge **estimator** and difference- based **estimator** through MSE and a MSE matrix. A **restricted** difference- based ridge **estimator**(13) was suggested to the semiparametric partial linear regression model, the necessary and sufficient conditions were also derived for a new

10 Read more

In machine learning research, many emerging applications can be (re)formulated as the composition optimization prob- lem with nonsmooth regularization penalty. To solve this problem, traditional **stochastic** gradient descent (SGD) algo- rithm and its variants either have low convergence rate or are computationally expensive. Recently, several **stochastic** composition gradient algorithms have been proposed, how- ever, these methods are still inefficient and not scalable to large-scale composition optimization problem instances. To address these challenges, we propose an asynchronous par- allel algorithm, named Async-ProxSCVR, which effectively combines asynchronous parallel implementation and variance reduction method. We prove that the algorithm admits the fastest convergence rate for both strongly convex and gen- eral nonconvex cases. Furthermore, we analyze the query complexity of the proposed algorithm and prove that linear speedup is accessible when we increase the number of pro- cessors. Finally, we evaluate our algorithm Async-ProxSCVR on two representative composition optimization problems in- cluding value function evaluation in reinforcement learn- ing and sparse mean-variance optimization problem. Exper- imental results show that the algorithm achieves significant speedups and is much faster than existing compared methods.

This section presents some numerical solutions of the controlled nonlinear system of the **stochastic** lattice gas of prey-predator model in Equation (18) and the estimators of the system unknown probabilities to show how the control for this system is possible. Also, numerical examples for controlled **stochastic** lattice gas of prey-predator model were carried out for various probabilities values and different initial densities. For illu- stration purpose, we display the numerical solutions of the system graphically. Fur- thermore, the percentage error in the estimate for real values of the parameters will be calculated. The percentage error PE p ˆ of the **estimator** p ˆ of the parameter p is giv-

16 Read more

We usually consider the generalization error in terms of a direct and an inverse problem. The direct problem involves solving the generalization error with a known true density function. The inverse problem is finding proper learning models and learning algorithms to minimize the gener- alization error under the condition of an unknown true density function. The inverse problem is important for practical usage, but in order to solve it, we first need to solve the direct problem. In this paper, we consider the direct problem of the **restricted** Boltzmann machine model.

30 Read more

and Strawderman [3] studied a shrinkage estimation of p positive normal means under sum of squared errors loss. Recently, Hoque et al. [10] investigated the performance of the shrinkage **estimator** of the parameters of a simple linear regression model under the asym- metric loss (LINEX loss criterion). For more details on this topic, we refer to Marchand and Strawderman [15], Silvapulle and Sen [18] and van Eeden [20], among others.

11 Read more

In the financial markets, sophisticated, complex products are continuously offered and traded. There are many financial products whose values depend on more than one underlying asset. Examples include basket options (options on the av- erage of several underlying assets), out-performance options (options on the maximum of several assets), spread options (options on the difference between two assets), and quan- tos (options whose payoff is adjusted by some **stochastic** variable, typically an exchange rate). Even when there is a single underlying asset, there is trend towards models with multiple **stochastic** factors (sources of uncertainty), e.g., single-asset model with **stochastic** volatility. In addition, multi-factor models are gaining more acceptance and use for modeling interest rates, where models with two to four factors are common and models with up to ten factors are being tested (Broadie and Glasserman 1997a). As comput-

Abstract: Principal component Analysis (PCA) is one of the popular methods used to solve the multicollinearity problem. Researchers in 2014 proposed an **estimator** to solve this problem in the linear model when there were **stochastic** linear restrictions on the regression coefficients. This **estimator** was called the **stochastic** **restricted** principal components (SRPC) regression **estimator**. The **estimator** was constructed by combining the ordinary mixed **estimator** (OME) and the principal components regression (PCR) **estimator**. It ignores the number of components (orthogonal matrix T r ) that the researchers

Stein-rule and ridge estimators have been extensively used for estimating the coefficient vector in a regression model. These estimators lead to an improvement in the risk properties of the ordinary least squares (OLS) **estimator**. Instead of using one or the other **estimator**, both of them may be appropriately combined. We introduce an alternative **estimator** that combines the approaches followed in obtaining the **restricted** Stein-rule estimation and the ridge regression estimation. A Monte Carlo simulation is performed to compare the behavior of the proposed **estimator**.

12 Read more

In this study, a common form of superiority conditions were obtained for com- parison among the biased estimators (RE, AURE, LE, AULE, PCRE, r-k class es- timator and r-d class **estimator**) and their predictors by using a generalized form for the misspecified linear regression model when explanatory variables are mul- ticollinearity. Furthermore, the theoretical findings were analyzed by using a numerical example and a Monte Carlo simulation study.

25 Read more