Instance-context embeddings

Though the learned word representations described in Section 2.5 has enjoyed much success they are actually founded on a false assumption, i.e. that each word has exactly one sense. This is clearly not true, e.g. the word rock may refer to either music or a stone.

In Paper VII, two approaches for computing sense aware learned word representations are introduced, where the first provide a baseline for the second approach. The baseline system constructs context dependent word representations by averaging the word repre- sentations described in Section 2.5.2 corresponding to the word tokens in their context. The drawback of the baseline approach, that we try to rectify in our second method Instance-Context Embeddings (ICE), is that it attends the same amount on all words in the context even though some words are clearly more indicative for deciding the sense of a given target word. Our solution to this problem is to attend more to the words to which the Skip-gram model assigns a high probability of occurring in the target words context. This means that the words that correlate with the target word will be attended to more, and that very common words that correlate with every word will be weighted less. This is due to the connection between the Skip-gram objective and pointwise mutual information showed in (Levy and Goldberg 2014), and has the effect of creating word representations that are more stable for word sense, see Figure 4.3.1, and less affected by the noise of unrelated words, e.g. stop words or words that are rarely used together with the target word.

(a) Uniform context embeddings (b) Instance-context embeddings (ICE) Medium Essay Scholarly article Newspaper Newspaper firm Material

Figure 4.3.1: Context representations for instances of the noun “paper” in the SemEval- 2013 test data, plotted using t-SNE. The legend refers to WordNet gold standard sense labels.

Chapter 5

Towards disentangled

representations

The increased performance in natural language processing, contributed to in Chapter 4 of this thesis, comes with a price in the form of decreased interpretability. Though, this might be a price that we are willing to pay in some instances there are situations in which interpretable learned word representations would be preferable, e.g. the ability to list all the properties encoded in a learned word representation would be invaluable for lexicographers. Though deep neural networks have the ability to learn representations that are increasingly abstract in deeper layers, disentangling the causes of variation in the underlying data (Bengio, Courville, and Vincent 2013), they suffer from their own power in that they are perfectly content using any arbitrarily chosen basis when encoding some property, e.g. plurality might be expressed along a line affecting all dimensions of the vector representing the word. Humans have a more limited capability when it comes to high dimensional reasoning and therefore prefer representations that represent independent factors of variation in separate dimensions of the representation, e.g. have a single dimension that encode plurality and nothing else. Another way to state this is that the representations are considered disentangled when unnecessary correlation between dimensions have been removed. Applying this notion to language, a recent paper by Subramanian et al. (2018) introduced a model for computing interpretable word representations. The model takes already trained word representations as input and use a denoising k-sparse autoencoder to learn interpretable versions them, i.e. learned word representations with disentangled dimensions.

Reflecting research question 4, defined in Section 1.2, the primary goals of the disen- tanglement approach presented in Paper VIII are:

• Increased interpretability of the individual dimensions in learned representations • Compatibility with gradient based neural network models

• Computational complexity on the same level as other neural network components.

5.1

The L

regularizer

The proposed solution to the goals defined above is a regularizer that actively penalizes covariance between dimensions of the representation, driving the model towards a more disentangled solution. This makes the model learn linearly uncorrelated representations which increases interpretability while obtaining good results on a number of tasks.

The regularizer is defined as

LΣ= 1 d2 N X i,j=1 |Cij|

where d is the dimensionality of the representation layer, and

C =_N1 − 1 N X i=1 (H− 1Nh)¯ T(H− 1Nh).¯

is the sample covariance of the representations in the representation layer over N examples. H = [h1; ...; hN] is a matrix of all representations in the batch, 1N is an N -dimensional

column vector of ones, and ¯h is the mean representation over the batch.

5.2

Selected results

Quantitative results on simulated data are presented in Figure 5.2.1 showing how the level of disentanglement influences the performance of a linear autoencoder, i.e. a neural network that is trained to recreate the input via an intermediate representation layer. The level of disentanglement is quantified using TdV which measures to what degree the total variance is captured inside the variance of the top d dimensions where d is equal to the actual dimension of the underlying data and UD90%which is the number of dimensions

needed to retain 90% of the total variance. The data is generated by sampling a d = 4 dimensional vector of independent features z_{∼ N(0, Σ), where Σ ∈ R}d×d _{is constrained}

to be non-degenerate and diagonal. However, before the data is fed to the autoencoder it is pushed through a random linear transformation x = Ωz. The goal of the model is to reconstruct properties of z in the representation layer while only having access to x.

The results using LΣ regularization are compared to results using L1regularization

which is known to lead to sparse solutions, i.e. representations where most dimensions are exactly zero. Though L1does not directly penalize correlation, maximally sparse solutions

can only be achieved if superfluous correlation between dimensions in the representation is removed. Hence, L1regularization can be seen as a crude tool for creating disentangled

representations and this is seen in the result where both L1 and LΣ produce disentangled

representations but LΣdoes so at a lower cost in performance drop.

Returning to the focus areas defined in the beginning of the section, it is clear that LΣ

regularization does improve the interpretability in the sense that fewer dimensions are used to encode the information, further evidence for this is presented in Paper VIII. Further, since the approach takes the form of an additive regularization term it can easily be added

Figure 5.2.1: The resulting dimensionality the representation layer after training the model with LΣ and L1 regularization respectively, measured in TdV (left) and UD90%

(right). The first point on each curve corresponds to λ = 0, i.e. no regularization, followed by 8 points logarithmically spaced between 0.001 and 1.All scores are averaged over 10 experiments using a different random projection (Ω).

to any gradient based model, and finally the computational complexity is O(N d2_{) which}

is on the same order as a single layer in a neural network (O(N d(current layer)_d(prev layer)₎₎

and can be efficiently computed on modern graphical processing units (GPU).

Further experiments, including disentangled representations of CIFAR-10 (Krizhevsky and G. Hinton 2009) images, is presented in Paper VIII.

Chapter 6

Concluding remarks

Following the questions posed in Section 1.2 this thesis has shown that multi-agent rein- forcement learning can be used to drive language evolution in a way that produces emerged language that partition semantic spaces in human-like ways. Though close correspondence to human languages remain a goal for future work, important properties like efficient communication were shown to consistently emerge and the presented frameworks could successfully be used to model language effects of environmental manipulation. Further, the semantics of the emerged symbols were shown to be grounded in the environment, e.g. symbols connected to colors as well as more other more abstract attributes.

In the context of learned word representations, several results were presented in Chapter 4 in support of research question 3, i.e. “Can learned word representations, computed using machine learning, be used to further improve the state-of-the-art in natural language processing?”, adding to the existing body of evidence showing that learned word representations can lead to clear improvements when used in natural language processing systems.

Finally, in Chapter 5 a first step was taken in the direction of learned representations with improved interpretability by showing that representations can be diagonalized by applying a covariance based regularizer, which makes it easier to connect individual dimensions to specific properties encoded in the representation, and ultimately makes the representation more interpretable.

6.1

Future work

A natural expansion to the communications frameworks presented in this theses would be to introduce multi-agent system with many participants. This would represent a more realistic setting and would make it possible to answer questions like, e.g. how term usage scales with the size of the population, the frequency and structure of inter-agent communication, and the lifespan of the individuals.

Another interesting avenue of research to follow is that of disentangled representations. In particular, an in-depth analysis of the implications of using the presented disentangle- ment method in the context of natural language, something that was left for future study

in the presented paper. Further, the interplay between disentangled representations and symbolic communication, which could be viewed as a form of disentanglement, would be interesting to pursue.

References

Abbott, J. T., T. L. Griffiths, and T. Regier (2016). Focal colors across languages are representative members of color categories. Proceedings of the National Academy of Sciences 113.40, 11178–11183.

Amrami, A. and Y. Goldberg (2018). “Word Sense Induction with Neural biLM and Symmetric Patterns”. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: Association for Computational Linguistics, pp. 4860–4867.

Baeza-Yates, R., B. Ribeiro-Neto, et al. (1999). Modern information retrieval. Vol. 463. ACM press New York.

Baker, C. F., C. J. Fillmore, and J. B. Lowe (1998). “The berkeley framenet project”. Proceedings of the 17th international conference on Computational linguistics-Volume 1. Association for Computational Linguistics, pp. 86–90.

Baroni, M., G. Dinu, and G. Kruszewski (2014). “Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors”. Proceedings of Association for Computational Linguistics (ACL). Vol. 1.

Bengio, Y., A. Courville, and P. Vincent (2013). Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence 35.8, 1798–1828.

Bengio, Y., R. Ducharme, et al. (2003). A neural probabilistic language model. journal of machine learning research 3.Feb, 1137–1155.

Bordes, A. et al. (2009). “Learning to Disambiguate Natural Language Using World Knowledge”. NIPS workshop on Grammar Induction, Representation of Language and Language Learning.

Cho, K. et al. (2014). “Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation”. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP, pp. 1724–1734.

Collobert, R. and J. Weston (2008). “A unified architecture for natural language processing: Deep neural networks with multitask learning”. Proceedings of the 25th international conference on Machine learning. ACM, pp. 160–167.

Faruqui, M. and C. Dyer (2015). “Non-distributional Word Vector Representations”. Pro- ceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). Beijing, China: Association for Computational Linguistics, pp. 464–469.

Ferrucci, D. A. (2012). Introduction to “this is watson”. IBM Journal of Research and Development 56.3.4, 1–1.

Foerster, J. N. et al. (2016). Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks. CoRR abs/1602.02672.

Foerster, J. et al. (2016). “Learning to Communicate with Deep Multi-Agent Reinforcement Learning”. Advances in Neural Information Processing Systems 29. Ed. by D. D. Lee et al., pp. 2137–2145.

Gibson, E., R. Futrell, et al. (2017). Color naming across languages reflects color use. Proc Natl Acad Sci USA 114.40, 10785–10790.

Gibson, E., S. T. Piantadosi, et al. (2013). A noisy-channel account of crosslinguistic word-order variation. Psychological science.

Greenberg, J. H. (2010). Language universals: With special reference to feature hierarchies. Walter de Gruyter.

Harnad, S. (1990). The symbol grounding problem. Physica D: Nonlinear Phenomena 42.1, 335–346.

Harris, Z. (1954). Distributional structure. Word 10.23, 146–162.

Havrylov, S. and I. Titov (2017). Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols. Workshop track at International Conference on Learning Representations (ICLR).

Hochreiter, S. and J. Schmidhuber (1997). Long short-term memory. Neural computation 9.8, 1735–1780.

Jorge, E., M. K˚ageb¨ack, F. D. Johansson, and E. Gustavsson (2016). “Learning to Play Guess Who? and Inventing a Grounded Language as a Consequence”. Proceedings of the NIPS workshop on deep reinforcement learning.

K˚ageb¨ack, M., D. Dubhashi, and A. Sayeed (2018a). “A reinforcement-learning approach to efficient communication”. In submission to PLOS One.

— (2018b). “DeepColor: Reinforcement Learning optimizes information efficiency and well-formedness in color name partitioning”. Proceedings of the 40th Annual Meeting of the Cognitive Science Society (CogSci).

K˚ageb¨ack, M., F. D. Johansson, R. Johansson, and D. Dubhashi (2015). “Neural context embeddings for automatic discovery of word senses”. Proceedings of the NAACL-HLT, pp. 25–32.

K˚ageb¨ack, M. and O. Mogren (2017). “Disentangled activations in deep networks”. Proceedings of the NIPS workshop on Learning Disentangled Features: from Perception to Control.

K˚ageb¨ack, M., O. Mogren, N. Tahmasebi, and D. Dubhashi (2014). “Extractive sum- marization using continuous vector space models”. Proceedings of the 2nd Workshop on Continuous Vector Space Models and their Compositionality (CVSC) at EACL, pp. 31–39.

K˚ageb¨ack, M. and H. Salomonsson (2016). “Word Sense Disambiguation using a Bidirec- tional LSTM”. Proceedings of the 5th Workshop on Cognitive Aspects of the Lexicon (CogALex). Association for Computational Linguistics.

Kay, P. and R. S. Cook (2014). World Color Survey. Encyclopedia of Color Science and Technology, 1–8.

Kemp, C., Y. Xu, and T. Regier (2018). Semantic Typology and Efficient Communication. Annual Review of Linguistics 4.1, 109–128.

Kirby, S., T. Griffiths, and K. Smith (2014). Iterated learning and the evolution of language. Current Opinion in Neurobiology 28. SI: Communication and language, 108–114.

Krizhevsky, A. and G. Hinton (2009). Learning multiple layers of features from tiny images. Technical report, University of Toronto.

Larsson, M., A. Nilsson, and M. K˚ageb¨ack (2017). “Disentangled representations for manipulation of sentiment in text”. Proceedings of the NIPS workshop on Learning Disentangled Features: from Perception to Control.

Lazaridou, A., A. Peysakhovich, and M. Baroni (2017). Multi-Agent Cooperation and the Emergence of (Natural) Language. Conference track at International Conference on Learning Representations (ICLR).

Levy, O. and Y. Goldberg (2014). “Neural Word Embedding as Implicit Matrix Factoriza- tion”. Advances in Neural Information Processing Systems 27. Ed. by Z. Ghahramani et al. Curran Associates, Inc., pp. 2177–2185.

Levy, O., Y. Goldberg, and I. Dagan (2015). Improving distributional similarity with lessons learned from word embeddings. Transactions of the Association for Computational Linguistics 3, 211–225.

Lin, H. and J. Bilmes (2011). “A Class of Submodular Functions for Document Summa- rization”. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. ACL, pp. 510–520.

Mikolov, T., K. Chen, et al. (2013). Efficient Estimation of Word Representations in Vector Space. ArXiv preprint arXiv:1301.3781.

Mikolov, T., W.-t. Yih, and G. Zweig (2013). “Linguistic regularities in continuous space word representations”. Proceedings of NAACL-HLT, pp. 746–751.

Miller, G. A. (1995). WordNet: a lexical database for English. Communications of the ACM 38.11, 39–41.

Mnih, V., K. Kavukcuoglu, D. Silver, A. Graves, et al. (2013). Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602.

Mnih, V., K. Kavukcuoglu, D. Silver, A. A. Rusu, et al. (2015). Human-level control through deep reinforcement learning. Nature 518.7540, 529.

Mogren, O., M. K˚ageb¨ack, and D. Dubhashi (2015). “Extractive Summarization by Aggregating Multiple Similarities”. Proceedings of Recent Advances in Natural Language Processing, pp. 451–457.

Mordatch, I. and P. Abbeel (2017). Emergence of Grounded Compositional Language in Multi-Agent Populations. arXiv preprint arXiv:1703.04908.

Pennington, J., R. Socher, and C. D. Manning (2014). Glove: Global vectors for word representation. Proceedings of the Empiricial Methods in Natural Language Processing (EMNLP 2014) 12, 1532–1543.

Peters, M. E. et al. (2018). Deep contextualized word representations. arXiv preprint arXiv:1802.05365.

Piantadosi, S. T., H. Tily, and E. Gibson (2011). Word lengths are optimized for efficient communication. Proceedings of the National Academy of Sciences 108.9, 3526–3529.

Rand, W. M. (1971). Objective criteria for the evaluation of clustering methods. Journal of the American Statistical association 66.336, 846–850.

Regier, T., P. Kay, and N. Khetrapal (2007). Color naming reflects optimal partitions of color space. Proc Natl Acad Sci USA 104.3, 1436–1441.

Regier, T., C. Kemp, and P. Kay (2015). “Word meanings across languages support efficient communication”. The handbook of language emergence. Ed. by B. MacWhinney and W. O’Grady. Hoboken NJ: Wiley-Blackwell., pp. 237–263.

Rumelhart, D. E., G. E. Hinton, and R. J. Williams (1986). Learning representations by back-propagating errors. nature 323.6088, 533.

Sahlgren, M. (Jan. 2006). “The Word-Space Model: Using distributional analysis to represent syntagmatic and paradigmatic relations between words in high-dimensional vector spaces”. PhD thesis. Stockholm University.

Sch¨utze, H. (1998). Automatic word sense discrimination. Computational linguistics 24.1, 97–123.

Silver, D., A. Huang, et al. (2016). Mastering the game of Go with deep neural networks and tree search. nature 529.7587, 484.

Silver, D., J. Schrittwieser, et al. (2017). Mastering the game of Go without human knowledge. Nature 550.7676, 354.

Socher, R., E. H. Huang, et al. (2011). “Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection”. Advances in Neural Information Processing Systems 24.

Socher, R., C. D. Manning, and A. Y. Ng (2010). “Learning continuous phrase repre- sentations and syntactic parsing with recursive neural networks”. Proceedings of the NIPS-2010 Deep Learning and Unsupervised Feature Learning Workshop.

Steels, L. (1995). A self-organizing spatial vocabulary. Artificial life 2.3, 319–332. Subramanian, A. et al. (2018). SPINE: SParse Interpretable Neural Embeddings. Proceed-

ings of the Thirty Second AAAI Conference on Artificial Intelligence (AAAI). Sukhbaatar, S., A. Szlam, and R. Fergus (2016). “Learning Multiagent Communication

with Backpropagation”. Advances in Neural Information Processing Systems 29 (NIPS). Ed. by D. D. Lee et al., pp. 2244–2252.

Tahmasebi, N., L. Borin, G. Capannini, D. Dubhashi, P. Exner, M. Forsberg, G. Gossen, F. D. Johansson, R. Johansson, M. K˚ageb¨ack, O. Mogren, P. Nugues, and T. Risse (2015). Visions and open challenges for a knowledge-based culturomics. International Journal on Digital Libraries 15.2-4, 169–187.

Williams, R. J. (1992). “Simple statistical gradient-following algorithms for connectionist reinforcement learning”. Reinforcement Learning. Springer, pp. 5–32.

Zaslavsky, N., C. Kemp, T. Regier, et al. (2018). Efficient compression in color naming and its evolution. Proceedings of the National Academy of Sciences.

Zaslavsky, N., C. Kemp, N. Tishby, et al. (2018). Color naming reflects both perceptual structure and communicative need. arXiv preprint arXiv:1805.06165.