[PDF] Top 20 Stochastic Methods for l1-regularized Loss Minimization

... similar to our method, with three main differences. First, and most important, at each iteration we choose a coordinate uniformly at random. This allows us to provide theoretical runtime guarantees. We note that no ... See full document

... An alternative approach to training a log-linear model is to use stochastic gradient descent (SGD) methods. SGD uses approximate gradients esti- mated from subsets of the training data and up- dates the ... See full document

... In large-scale machine learning applications for big data analysis, it becomes a common practice to partition the training data and store them on multiple machines connected via a commodity network. A typical setting of ... See full document

... optimization methods for least-square ...least-square loss, the minimization of the one-variable sub-problem (11) has a closed-form ...descent methods, although they allowed a block of ... See full document

... Newton methods are known to have fewer iterations although each iteration costs ...the loss calculation, this type of methods may surpass CDN under some ...In L1-regularized ... See full document

... gradient methods enjoy the improved iteration complexity O((1 + √ κ) log(1/)) (Nesterov, 2004; Tseng, 2008; Beck and Teboulle, 2009; Nesterov, 2013) 1 ...batch methods requires a full pass over the dataset, ... See full document

... detection methods were introduced in power system networks demonstrating self- healing ...for loss minimization using an iterative-analytical ...the stochastic nature of some of the ... See full document

... of regularized empirical risk minimization based on con- vex loss functions plays an important role, see Vapnik ...such methods is that many classifiers based on convex loss functions ... See full document

... gradient methods (including mirror-descent methods) in which the new iterate is obtained by stepping from the current iterate along a single subgradient, and then followed by a ...based stochastic ... See full document

... following methods have been applied to several problems (Hastie et ...general regularized convex loss minimization ...Path-following methods can be slow for large-scale problems, where ... See full document

... The outline of our paper is as follows. In Section 2 we describe BMRM and contrast it with stan- dard bundle methods. We also prove rates of convergence. In Section 3 we discuss implementation issues and present ... See full document

... In Figure 5 we compare choosing dual variables at random with repetitions (as done in SDCA) vs. choosing dual variables using a random permutation at each epoch (as done in SDCA-Perm) vs. choosing dual variables in a ... See full document

... In this section, we present the following concrete examples to judge the numerical per- formance of our algorithm. By using the algorithm in Theorem . and Theorem ., we illustrate its realization, effectiveness, and ... See full document

... Detailed analysis of the results shows TRegMT score achieves better N -gram match percentages than Moses translation but suffers from the brevity penalty due to selecting shorter translations. Due to using a cost ... See full document

... both methods are useful for detecting current CD, they do not provide predictive information on the future risk of developing CD in a person without active ... See full document

... to the problem of reconstructing essentially piecewise constant medical images in computerized tomography (CT) in the compressed sensing framework via total variation minimization. Tested with the Shepp-Logan ... See full document

... Probably since one can solve a multiclass classification problem (K > 2) by solving several binary classification problems, there are much fewer studies on multiclass classification algorithms based directly on ... See full document

... CG methods can achieve the optimal convergence rate for first-order methods, 3 but for non-quadratic functions, the situation is ...first-order methods for a specific choice of α k and β k ... See full document

... In this paper, we study the sparsest solutions of linear complementarity problems (LCPs), which study has many applications, such as bimatrix games and portfolio selections. Mathematically, the underlying model is ... See full document

... The rest of the paper is organized as follows. In Section , we summarize the existing methods for solving the constrained problem (.). In Section , the generalized shrinkage operator is proposed. The new ... See full document