unsupervised learning

Top PDF unsupervised learning:

Unsupervised Learning for Persian WordNet Construction

Unsupervised Learning for Persian WordNet Construction

In this paper we introduce an unsupervised learning approach for WordNet construction. The whole construction method is an Expecta- tion Maximization (EM) approach which uses Princeton WordNet 3.0 (PWN) and a corpus as the data source for unsupervised learning. The proposed method can be used to construct WordNet in any language. Links between PWN synsets and target language words are extracted using a bilingual dictionary. For each of these links a parameter is defined that shows probability of selecting PWN synset for target language word in corpus. Model para- meters are adjusted in an iterative fashion. In our experiments on Persian language, by se- lecting 10% of highly probable links trained by the EM method, a Persian WordNet was obtained that covered 7,109 out of 11,076 dis- tinct words and 9,427 distinct PWN synsets with a precision of more than 86%.
Show more

7 Read more

Automatic Text Categorization by Unsupervised Learning

Automatic Text Categorization by Unsupervised Learning

In this paper, we propose a new automatic text categorization method based on unsupervised learning. Without creating training documents by hand, it automatically creates training sentence sets using keyword lists of each category. And then, it uses them for training and classifies text documents. The proposed method can provide basic data for creating training documents from collected documents, and can be used in an application area to classify text documents in low cost. We use the χ 2

7 Read more

Unsupervised learning for image classification

Unsupervised learning for image classification

Unsupervised Learning, Convolutional Neural Networks, Deep Learning, Image Classication This thesis is an investigation of unsupervised learning for image classication. The state-of-the-art image classication method is Convolutional Neural Network (CNN), which is a purely supervised learning method. We argue that despite of the triumph of supervised learning, unsupervised learning is still important and compatible with supervised learning. For example, in the situation where some classes have no training data at all, so called zero-shot learning task, unsupervised learning can leverage supervised learning to classify the images of unseen classes. We proposed a new zero- shot learning method based on CNN and several unsupervised learning algorithms. Our method achieves the state-of-the-art results on the largest public available labelled image dataset, ImageNet fall2011.
Show more

55 Read more

Unsupervised Learning of Narrative Schemas and their Participants

Unsupervised Learning of Narrative Schemas and their Participants

JUDGE , SUSPECT )) whose arguments are filled with participant semantic roles defined over words (J UDGE = {judge, jury, court}, P OLICE = {police, agent, authorities}). Unlike most previous work in event structure or semantic role learning, our sys- tem does not use supervised techniques, hand-built knowledge, or predefined classes of events or roles. Our unsupervised learning algorithm uses corefer- ring arguments in chains of verbs to learn both rich narrative event structure and argument roles. By jointly addressing both tasks, we improve on pre- vious results in narrative/frame learning and induce rich frame-specific semantic roles.
Show more

9 Read more

Improving unsupervised learning with exemplarCNNs

Improving unsupervised learning with exemplarCNNs

Other approaches step outside the unsupervised learning paradigm and try to maximally leverage the small set of labeled samples using techniques such as extreme data augmentations (DeVries and Taylor, 2017; Zhong et al., 2017), clustering techniques (Ji and Vedaldi, 2018; Dundar et al., 2015), autoencoders (Zhao et al., 2015), or other more theoretical approaches (Hjelm et al., 2019; Ji and Vedaldi, 2018). In (Hjelm et al., 2019) convolutional k-means clustering is proposed as a technique to exploit the relationships between the layers from a deep convolutional neural network to prevent the network from learning redundant filters and learn more efficiently from less labeled data. Regarding autoencoders, (Zhao et al., 2015) propose a method where each pooling layer from the encoder generates a variable with image content information that passes to the following layer, and a variable with spatial information that passes to the corresponding layer in the decoder. (Hjelm et al., 2019) propose a more theoretical approach based on the maximization of the mutual information between the output of an encoder CNN and local regions of the input. Similarly, (Ji and Vedaldi, 2018) minimize the mutual information between the classes predicted for a pair of patches extracted of two images.
Show more

8 Read more

Unsupervised Learning of the Morphology of a Natural Language

Unsupervised Learning of the Morphology of a Natural Language

This study reports the results of using minimum description length MDL analysis to model unsupervised learning of the morphological segmentation of European languages, using corpora rang[r]

46 Read more

Unsupervised Learning of Generalized Names

Unsupervised Learning of Generalized Names

Unsupervised Learning of Generalized Names Roman Yangarber, Winston Lin, Ralph Grishman Courant Institute of Mathematical Sciences New York University froman|winston|grishmang@cs nyu edu Abstract We p[.]

7 Read more

Unsupervised Learning from Shollow to Deep

Unsupervised Learning from Shollow to Deep

Unsupervised learning is widely used in many computer vision applications from low-level tasks, such as image denoising (Mairal et al. [2009b]; Sulam et al. [2014]), in-painting (Peyré [2009]; Roth and Black [2009]) and dimension reduction (Comon [1994]), to high-level tasks, such as image clustering (Xie et al. [2016]) and feature learning (Kingma and Welling [2013]; Hjelm et al. [2018]). Unsupervised learning is often used to learn the underlying represen- tation of input data, and its techniques are evolving all the time. The most well-known al- gorithms, such as Principal Component Analysis (PCA) (Wold et al. [1987]), Independent Component Analysis (ICA) (Comon [1994]), have been introduced to reduce the dimension of input to find concise lower dimension representation. Afterward, sparse representation (Ol- shausen and Field [1997]; Aharon et al. [2006]) has attracted huge attention from researchers and it has formed a complete theoretical system and application methods, where the over- complete dictionary or basis could be deterministic or data-driven. Nowadays, Convolution Neural Networks (CNN) have become the most popular framework for researchers to conduct their research. Compared to the development of supervised learning, there are fewer studies on training CNN models without supervision. Deep Boltzmann Machines (DBM) (Salakhut- dinov and Hinton [2009]; Salakhutdinov and Larochelle [2010]) and Deep Belief Networks (DBN) (Boureau et al. [2008]; Lee et al. [2009]; Hinton [2009]) were proposed in unsuper- vised learning to learn feature representation. After demystifying several fundamental tech- niques such as saturating gradient (Glorot et al. [2011]) which help training CNNs much easier, unsupervised learning has been successfully applied in many downstream vision and language tasks Hjelm et al. [2018]; Yang et al. [2017]; Ji et al. [2017b].
Show more

140 Read more

Unsupervised learning of shape manifolds

Unsupervised learning of shape manifolds

Let us regard a shape as a closed curve (contour) and initially represent it as a set of boundary points corresponding to the contour. We have chosen Fourier descriptors to represent a given shape contour. Fourier descriptors are simple yet powerful means of extracting shape features. It is worth noting that the choice of Fourier descriptors was made with simplicity in mind. The major objective of this study is to investigate the unsupervised learning of shape manifolds; no claim of optimality is made here about the representation of shapes using Fourier descriptors. Having noted that, one of the advantages of using Fourier descriptors is that they can easily be made invariant to any scale and rotation transformations (as described in the next paragraph). This implies that no explicit shape normalisation, as is the case with most well-known shape analysis methods such as [4], is required.
Show more

11 Read more

Localized Feature Selection For Unsupervised Learning

Localized Feature Selection For Unsupervised Learning

On the above experiments, the number of clusters k is set to the “true” number of classes. This is not always applicable in real world applications. How to determine the value of k is a common problem in unsupervised learning. It may strongly interact with the predicted clus- ter structures, as well as the selected feature subset in feature selection algorithms [1]. There are several algorithms available to determine k, i.e., [1, 43, 71]. Another common problem that a clustering algorithm usually faces is how to initialize cluster centroids. Bad initial clus- ters/centroids might lead to low quality clusters. In traditional clustering algorithms, some techniques, such as randomly picking up k patterns over the dataset, preliminary clustering, or choosing the best from several iterations, are frequently used to alleviate the chance of bad initial clusters. In our approach, bad initial clusters for backward searching may occur more often when many noise features presented, and might affect the final clusters and feature sub- sets largely. This problem can be alleviated by preliminary clustering with a global feature selection, i.e., [1].
Show more

99 Read more

Unsupervised Learning with Contrastive Latent Variable Models

Unsupervised Learning with Contrastive Latent Variable Models

In unsupervised learning, dimensionality reduction is an im- portant tool for data exploration and visualization. Because these aims are typically open-ended, it can be useful to frame the problem as looking for patterns that are enriched in one dataset relative to another. These pairs of datasets occur com- monly, for instance a population of interest vs. control or sig- nal vs. signal free recordings. However, there are few meth- ods that work on sets of data as opposed to data points or sequences. Here, we present a probabilistic model for dimen- sionality reduction to discover signal that is enriched in the target dataset relative to the background dataset. The data in these sets do not need to be paired or grouped beyond set membership. By using a probabilistic model where some structure is shared amongst the two datasets and some is unique to the target dataset, we are able to recover interesting structure in the latent space of the target dataset. The method also has the advantages of a probabilistic model, namely that it allows for the incorporation of prior information, handles missing data, and can be generalized to different distribu- tional assumptions. We describe several possible variations of the model and demonstrate the application of the technique to de-noising, feature selection, and subgroup discovery set- tings.
Show more

8 Read more

A Multilinear Approach to the Unsupervised Learning of Morphology

A Multilinear Approach to the Unsupervised Learning of Morphology

It is well-known that Semitic languages pose prob- lems for the unsupervised learning of morphol- ogy (ULM). For example, Hebrew morphology exhibits both agglutinative and fusional processes, in addition to non-concatenative root-and-pattern morphology. This diversity in types of morpho- logical processes presents unique challenges not only for unsupervised morphological learning, but for morphological theory in general. Many previ- ous ULM approaches either handle the concatena- tive parts of the morpholgy (e.g., Goldsmith, 2001; Creutz and Lagus, 2007; Moon et al., 2009; Poon et al., 2009) or, less often, the non-concatenative parts (e.g., Botha and Blunsom, 2013; Elghamry, 2005). We present an approach to clustering morphologically related words that addresses both concatenative and non-concatenative morphology via the same learning mechanism, namely the Multiple Cause Mixture Model (MCMM) (Saund, 1993, 1994). This type of learning has direct con- nections to autosegmental theories of morphology
Show more

10 Read more

Unsupervised Learning on an Approximate Corpus

Unsupervised Learning on an Approximate Corpus

Most of this work has used the Penn Treebank (Marcus et al., 1993) as a dataset. While this million-word Wall Street Journal (WSJ) corpus is one of the largest that is manually annotated with parts of speech, unsupervised learning methods could take advantage of vast amounts of unannotated text. In practice, runtime concerns have sometimes led researchers to use small subsets of the Penn Tree- bank (Goldwater and Griffiths, 2007; Smith and Eis- ner, 2005; Haghighi and Klein, 2006). Our goal is to point the way to using even larger datasets.

11 Read more

Feature Selection for Unsupervised Learning

Feature Selection for Unsupervised Learning

A significant body of research exists on methods for feature subset selection for supervised data. These methods can be grouped as filter (Marill and Green, 1963; Narendra and Fukunaga, 1977; Almuallim and Dietterich, 1991; Kira and Rendell, 1992; Kononenko, 1994; Liu and Setiono, 1996; Cardie, 1993; Singh and Provan, 1995) or wrapper (John et al., 1994; Doak, 1992; Caruana and Freitag, 1994; Aha and Bankert, 1994; Langley and Sage, 1994; Pazzani, 1995) approaches. To maintain the filter/wrapper model distinction used in supervised learning, we define filter methods in unsupervised learning as using some intrinsic property of the data to select features without utilizing the clustering algorithm that will ultimately be applied. Wrapper approaches, on the other hand, apply the unsupervised learning algorithm to each candidate feature subset and then evaluate the feature subset by criterion functions that utilize the clustering result.
Show more

45 Read more

Painless Unsupervised Learning with Features

Painless Unsupervised Learning with Features

Unsupervised learning methods have been increas- ingly successful in recent NLP research. The rea- sons are varied: increased supplies of unlabeled data, improved understanding of modeling methods, additional choices of optimization algorithms, and, perhaps most importantly for the present work, in- corporation of richer domain knowledge into struc- tured models. Unfortunately, that knowledge has generally been encoded in the form of conditional independence structure, which means that injecting it is both tricky (because the connection between independence and knowledge is subtle) and time- consuming (because new structure often necessitates new inference algorithms).
Show more

9 Read more

Unsupervised Learning of Morphology

Unsupervised Learning of Morphology

As we said in Section 2, there is an explicit expectation frequently encountered in the more recent literature that ULM and other unsupervised methods could be employed in order to rapidly and cheaply (in terms of human effort) bootstrap basic language technology resources for new languages. However, looking at the literature, it seems that—at least in the area of inflectional morphology—the only approaches that have so far produced substantial results are the old-fashioned, hand-coded grammar-based ones, such as the work described by Trosterud (2004), where finite-state morphological processors and constraint grammar-based disambiguation components are developed for a number of related languages. The fact that the languages are related is of great help when dealing with successive languages after the first one. The morphological component for the first language, North Sámi, required approximately 2.5 person-years of highly qualified linguistic expert work to reach the prototype stage, whereas the analogous module for the closely related Lule Sámi was completed in an additional six months (Trosterud 2006). 17 This and other work in the same vein reported in the literature (e.g., by Artola-Zubillaga 2004 and Maxwell and David 2008) is characterized by deep and long-lasting involvement by linguistic expertise and further often by the creative use of digitized versions of conventional printed linguistic resources, especially dictionaries. The following observation is perhaps trivial, but bears stressing, because it is in fact often not heeded in practice: For this kind of approach to work, it is necessary that tools for providing systems with linguistic knowledge use a conceptual apparatus and notation familiar to the linguists who are supposed to be working with them. Relevant to our purposes here, the same holds for any attempt to kickstart the development of a morphological analyzer by using ULM: If the expectation is that the output of ULM should be manually “post-edited,” this output must of course be intelligible to the linguist doing the post-editing.
Show more

42 Read more

Unsupervised Learning of Semantic Relation Composition

Unsupervised Learning of Semantic Relation Composition

This paper presents an unsupervised method for deriving inference axioms by composing semantic relations. The method is indepen- dent of any particular relation inventory. It relies on describing semantic relations using primitives and manipulating these primitives according to an algebra. The method was tested using a set of eight semantic relations yielding 78 inference axioms which were eval- uated over PropBank.

10 Read more

Unsupervised Learning of Distributional Relation Vectors

Unsupervised Learning of Distributional Relation Vectors

We have proposed an unsupervised method which uses co-occurrences statistics to represent the re- lationship between a given pair of words as a vec- tor. In contrast to neural network models for rela- tion extraction, our model learns relation vectors in an unsupervised way, which means that it can be used for measuring relational similarities and related tasks. Moreover, even in (distantly) super- vised tasks (where we need to learn a classifier on top of the unsupervised relation vectors), our model has proven competitive with state-of-the-art neural network models. Compared to approaches that rely on averaging word vectors, our method is able to learn more faithful representations by fo- cusing on the words that are most strongly related to the considered relationship.
Show more

11 Read more

Unsupervised Learning of PCFGs with Normalizing Flow

Unsupervised Learning of PCFGs with Normalizing Flow

layers with q (i) being a feed-forward network with one hidden layer for both NICE and Real NVP, following He et al. (2018). We train the system until the marginal likelihood over the whole train- ing set starts to oscillate, around 10,000 batches for smaller corpora and around 20,000 for larger corpora. Because the inside algorithm is quadratic on the length of the sentences, the batch size for training gets quadratically smaller from 400 to 1 as sentences get longer. We use the Adam optimizer (Kingma and Ba, 2015), initialized with learning rates 0.1 for d and N, and 0.001 for L and parame- ters in g −1 . Means and standard deviations of eval- uation metrics are reported in tables with 10 runs of the proposed system.
Show more

11 Read more

Topics in unsupervised learning

Topics in unsupervised learning

Clustering based on m ixture models has appeared in the literature with increased frequency in recent years. The approach followed herein involves a family of Gaussian m ixture models w ith parsimonious covariance structure. The clustering of the data, according to this family of models, is given by the values assigned to the group membership labels through the learning process. Well-established m ixture modelling techniques are used for motivation and the m athem atical foundation of this family of models is established. These models are then applied to real d ata and perform favorably when compared to well-established techniques. A family of Gaussian m ixture models th a t can be applied to longitudinal d ata is also introduced.
Show more

174 Read more

Show all 10000 documents...