Computational models of language emergence – what is still missing?

Julian Zubek, Joanna Rączaszek-Leonardi

Abstract

Last 10 years have seen a proliferation of computational models of language emergence (Lazaridou & Baroni, 2020). Despite their variety (in concerns and architectures) they still seem to cling to several key assumptions about the communication process. We try to identify those among the assumptions which may hinder progress and point to possible ways to substitute them with less-limiting ones. We discuss three key issues.

First, in the existing models environments tend to be extremely simplified, often reduced to a set of static stimuli presented to a passive agent. Agent’s actions are limited to choosing the target stimulus from the set of distractors. There is no structure of agent’s actions which could guide the structure of emerging communication.

Second, the function of communication is often limited to describing the properties of the environment. The communication is unidirectional: there are distinguished roles of speaker and listener. What is missing is the interactivity: there is no recognition that interaction itself is a meaning-making process (De Jaegher & Di Paolo, 2007). Communication can be about (regulating) the interaction, rather than about prespecified “meanings”.

Third, signals are often cast as amodal, discrete symbols devoid of any physical properties. Signals are not a part of the environment in a meaningful way. Existence of discrete symbols with replicable form is presupposed rather than explained (Rączaszek-Leonardi, 2012). Such symbols can be composed in an arbitrary manner and assigned meanings equally arbitrary. There are no external constraints to guide their interpretation

Under such assumptions, the meaning of signals is reduced to static mapping from symbols to concepts. We propose to move away from this simplification, and orient towards dynamic and pragmatic aspects of communication. We discuss how this reorientation helps to conceptualize new models of language emergence.

Literature

• Lazaridou, A., & Baroni, M. (2020). Emergent Multi-Agent Communication in the Deep Learning Era. ArXiv:2006.02419 [Cs]. http://arxiv.org/abs/2006.02419
• De Jaegher, H., & Di Paolo, E. (2007). Participatory sense-making. Phenomenology and the Cognitive Sciences, 6(4), 485–507. https://doi.org/10.1007/s11097-007-9076-9
• Rączaszek-Leonardi, J. (2012). Language as a System of Replicable Constraints. In H. H. Pattee & J. Rączaszek-Leonardi, LAWS, LANGUAGE and LIFE (Vol. 7, pp. 295–333). Springer Netherlands.

— What do you think? Questions: