Sounding out ideas on language, interaction, and iconicity

Mark Dingemanse

Social robots as depictions of social agents

Clark &amp; Fischer propose that people see social robots as interactive depictions and that this explains some aspects of people’s behaviour towards them. We agree with C&amp;F’s conclusion that we don’t need a novel ontological category for these social artefacts and that they can be seen as intersecting with a lineage of depictions from Michelangelo’s David to Mattel’s talking barbie doll. We have two constructive contributions to make.

How robots become social: A comment on Clark &amp; Fischer

Sketches, visualizations and other forms of externalizing cognition play a prominent role in the work of just about any scientist. It’s why we love using blackboards, whiteboards, notebooks and scraps of paper. Many folks who had the privilege of working the late Pieter Muysken fondly remember his habit of grabbing any old piece of paper that came to hand, scribbling while talking, then handing it over to you.

Thinking visually with Remarkable

I have other things to do but one day I’ll enlarge on the insidious effects of elevating this cursed little histogram of “Research output per year” as the single most important bit of information about academics at thousands of universities that use Elsevier Pure. Consider this mini-rant my notes for that occasion. Most importantly, we DO NOT write per year. Our careers are too diverse and precarious to measure output that way.

A rant about Elsevier Pure

The closing paragraphs of my previous post were cited in several places (e.g. Culture Making, Far Outliers) as evidence of a cultural revival. Although I feel it is really too soon to say whether this is the case, I’m glad to report that the dirges that we recorded in Akpafu-Todzi are in wide circulation now and are even being played during funerals, to great acclaim.

A cultural revival?

Today’s dish of expressive vocabulary is particularly tasty. It comes from G|ui, a Khoisan language of Botswana.

 1

To Africanists, expressive words from Khoisan languages are of special interest because Khoisan has been claimed on various occasions to lack ideophones, otherwise thought to be one of those linguistic traits that characterize Africa as a linguistic area (Meeussen 1975:3,

 2

Heine &amp;

Fresh wild melon and meat full of gravy: food texture verbs in G|ui (Khoisan)

Language makes us human. But there is an interesting asymmetry in our willingness to ascribe linguistic capacities to non-humans: animals are seen as having none, whereas computers may well master it according to many. What curious conception of language makes this asymmetry possible? And what do Descartes and Turing have to do with it? Notes from a new essay about language between animals and computers.

Language is what makes us human: one of those things, perhaps

 the

one thing, that sets us apart. But there is an interesting asymmetry in our willingness to ascribe linguistic capacities to non-humans: animals tend to be seen as having none, whereas computers are increasingly thought to have mastered language. This asymmetry is the focus of a recent essay I co-authored with a range of people, led by Marlou Rasenberg.

Language between animals and computers

The last time I blindly accepted an invitation to speak was in 2012, when I was invited to an exclusive round table on the future of linguistics at a renowned research institute. As a fresh postdoc I was honoured and bedazzled. When the programme was circulated, I got a friendly email from a colleague asking me how I’d ended up there, and whether I thought the future of linguistics was to be all male.

How to avoid all-male panels (manels)

This is a the second part in a two part series of peer commentary on a recent preprint. The first part is here. I ended that post by noting I wasn’t sure all preprint authors were aware of the public nature of the preprint. I am now assured they are, and have heard from the senior author that they are working on a revised version.

Pitfalls of fossil-thinking part II

A lot of our recent work revolves around working with conversational data, and one thing that’s struck me is that there are no easy ways to create compelling visualizations of conversation as it unfolds over time. The most common form seems to be pixelated screenshots of transcription software not made for this purpose.

Playing with R: unrolling conversation

The construction of gothic cathedrals like Chartres was governed not by blueprints but by “talk, tradition, and templates” — at least that is what Turnbull has compellingly argued. When you come across such a neatly alliterative triad, there are two ways you can go. You can adopt the terms in an unexamined way and rely on their alliterative power.

Reading Suchman’s classic

 Human-machine reconfigurations: plans and situated actions

, I am impressed by her description David Turnbull’s work on the construction of gothic cathedrals. In brief, the intriguing point is that no blueprints or technical drawings or even sketches are known to have existed for any of the early modern gothic cathedrals, like that of Chartres.

Talk, tradition, templates: a meta-note on building scientific arguments

There is a considerable halo-effect attached to JIFs, whereby an article that ends up in a high IF journal (whether by sheer brilliance or simply knowing the right editor, or both) is treated, unread, with a level of veneration normally reserved for Wunderkinder. Usually this is done by people totally oblivious to network effects, gatekeeping and institutional biases.

I’ve been caught up in a few debates recently about

 Recognition and Rewards

, a series of initiatives in the Netherlands to diversify the ways in which we recognize and reward talent in academia. One flashpoint was the publication of an open letter signed by ~170 senior scientists (mostly from medical and engineering professions), itself written in response to two developments.

Why article-level metrics are better than JIF if you value talent over privilege

Wikidata is an ambitious enterprise, but social ontologies are never language-agnostic — so the project risks perpetuating rather than transcending the worldviews most prevalent in current Wikipedia databases, which means broadly speaking global north, Anglo, western, white cishet male worldviews. I think Wikidata is perhaps promising for brute physical facts like the periodic table and biochemistry.

TL;DR

Wikidata is an ambitious enterprise, but social ontologies are never language-agnostic — so the project risks perpetuating rather than transcending the worldviews most prevalent in current Wikipedia databases, which means broadly speaking global north, Anglo, western, white cishet male worldviews. I think Wikidata is perhaps promising for brute physical facts like the periodic table and biochemistry.

Concrete reasons to be skeptical about an ‘Abstract Wikipedia’

After much postponement, writing the final report for my NWO Veni grant (2015-2018) turned out to be an unexpected pleasure. It made me realise a couple of things — key among them the role of serendipity in shaping fundamental research. The project was called “Towards a science of linguistic depiction”. Looking at the publications that came out of it and at imperfect indicators like citations, it’s made some useful contributions.

The role of serendipity in shaping fundamental research

Magritte’s best known work by far is of course his drawing of a pipe with the text

 Ceci n’est pas une pipe

. He made several versions over the years, but the work originated in 1928 or 1929. The title Magritte gave to this painting is

 La trahison des images

— the treachery of images.

Magritte on Words and Images (PDF)

Ideophones, like so many things in life, are easy to identify but hard to define. Many researchers have grumbled about the shortcomings of Doke’s descriptive characterization of ideophones (see discussion here), but few have attempted to formulate an alternative. For better or worse, I did,

 1

but it took me a few iterations to arrive at something that I felt worked well enough to be useful in cross-linguistic research.

What’s wrong with “vivid”? The evolution of a definition

Few historical maps of Ghana’s Volta and Oti regions have been invested with so much political and sociohistorical meaning as Hans Gruner’s 1913 map of the Togo Plateau. Gruner, stationed for over twenty years at Misahöhe in present-day Togo, was a long-time colonial administrator known for his ethnographical and historical knowledge of the area.

The Gruner Map: a 1913 map of the Togo Plateau in present-day Ghana

DALL-E, a new image generation system by OpenAI, does impressive visualizations of biased datasets. I like how the first example that OpenAI used to present DALL-E to the world is a meme-like koala dunking a baseball leading into an array of old white men — representing at one blow the past and future of representation and generation.

Deep learning, image generation, and the rise of bias automation machines

I have been blogging at

 The Ideophone

since 2007, and not all of it has been as ephemeral as my PhD promotor once feared. My short post documenting the etymology of Zotero is apparently the only scientific documentation of where Zotero’s name comes from; it has served as a source in Wikipedia for ages and has received over 15 scholarly citations.

Scholarly blogging, now with DOIs

Academia.edu takes your academic work and puts it behind a privacy-defying signup form, laces it with ads, botches the metadata, and tries to appease you by offering stats and social ranking that promote constant comparison. What’s not to like?

A while back some low quality citations started showing up on Google Scholar. They had titles like “CHAPTER 2 draft — email xyz@ab.edu” and it was hard find actual bibliographic metadata. Google Scholar seemed to have scraped random PDFs uploaded on Academia.edu and decided it was worth counting the citations in them even in the absence of proper metadata.

How Academia.edu promotes poor metadata and plays to our vanity

Interjections are, in Felix Ameka’s memorable formulation, “the universal yet neglected part of speech” (1992). They are rarely the subject of historical, typological or comparative research in linguistics, and as Aimée Lahaussois has shown (2016), they are notably underrepresented in descriptive grammars. As grammars are the main source of data for typologists, this is of course a perfect example of a self-reinforcing feedback loop.

How should descriptive grammars cover interjections?

Lezenswaardig: een groep jonge medici ageert tegen de marketing-wedstrijd waarin volgens hen narratieve CVs in kunnen ontaarden — de nieuwste bijdrage aan het Erkennen &amp; Waarderen-debat. Maar niets is wat het lijkt. Over evidence-based CVs, kwaliteit &amp; kwantificatie.

Eerst dit: de brief benoemt het risico dat je met narratieve CVs een soort competitie krijgt tussen verhalen.

Van betekenisloze getallen naar een evidence-based CV

Large language models make it entirely trivial to generate endless amounts of seemingly plausible text. There’s no need to be cynical to see the virtual inevitability of unending waves of algorithmically tuned AI-generated uninformation: the market forces are in place and they will be relentless.

Large language models and the unstoppable tide of uninformation

With Times Higher Education writing about citation gaming and hyperprolific authors (surely not unrelated) I hope we can save some of our attention for what Uta Frith and others have called slow science.

Team science is slow science

PLOS ONE notoriously and astonishingly does not have a proofs stage: authors do not get to see how their work is typeset until the very day it appears. Worse, they have a policy of only correcting formatting errors that they themselves introduce by issuing a formal correction notice — a heavyhanded method that most other journals use only for serious errors of fact or process. Unnecessary corrections hurt authors.

Note: I prepared this posting in August 2015, when PLOS ONE was due to publish a paper by us and I wanted to make sure they’d avoid the stupid typesetting errors they made in our 2013 paper. I used the numbers to convince them to show us proofs beforehand.

Why PLOS ONE needs page proofs

Travel journals provide some of the first written sources on Akpafu. I have previously posted an excerpt from a 1887 journal by David Asante. This here is an excerpt from a similar journey two years later. The whole journey took three months, but this excerpt relates only a trip to two Akpafu towns on 17-18 December 1889.

A visit to Akpafu by Nicolas Clerk, 1889

Sigismund Wilhelm Koelle is one of the founding fathers of African linguistics, and 1854 was one of his more productive years: he published the first large-scale comparison of some 200 African languages (the famed Polyglotta Africana), but also a corpus of Kanuri folklore, a grammar of Vai, and a grammar of Kanuri.

It is high time for a continuation of our series honouring the ancestors of ideophone studies. Sigismund Wilhelm Koelle is one of the founding fathers of African linguistics, and 1854 was one of his more productive years.

Early sources on African ideophones, part IV: S.W. Koelle on Kanuri, 1854

Zotero takes it name from an Albanian verb minus some morphology.

If you’ve read yesterday’s post (Zotero, an Endnote alternative) or come across Zotero elsewhere, you may have been wondering about its name.

The etymology of Zotero

The most common use of the term ‘ideophone’ today is as a term for a lexical or grammatical category of words. This use goes back to C.M. Doke, who introduced the term in this sense for the description of Bantu languages. However, Doke did not coin the term. Who did, and how was it used before Doke?

(

 Note:

Looking for a modern definition of the term? Check out “

 ‘Ideophone’ as a comparative concept

” (2019). That chapter supplies the following:

 Ideophone

*. Member of an open lexical class of marked words that depict sensory imagery.* It provides evidence and arguments for the cross-linguistic utility of this definition.) A common term for expressive vocabulary in African linguistics is ‘ideophone’.

On the history of the term ‘ideophone’

Some years ago I was following a course by Maarten Kossmann on Tuareg (Tamasheq, Tamajeq, Tamahaq). It was thoroughly enjoyable. After the first lecture we were all alotted a letter of the great

 Dictionnaire Touareg — Français


 1

(a consonant, obviously), and for the remainder of the course these 15 to 40 dictionary pages would form the basis for a number of excercises in the weeks to follow. It was an interesting method;

Expressive verbs in Tuareg

I note with sadness that William J. Samarin has passed away in Toronto on January 16, 2020 at the age of 93. An all too short obituary notes that he was “known for his work on the language of religion and on two Central African languages: Sango and Gbeya”.  In linguistics, Samarin was of course also known for his extensive work on ideophones, playful and evocative words with sensory meanings.

Farewell, Mr. Ideophone: William J. Samarin (1926-2020)

There is a minor industry in speech science and NLP devoted to detecting and removing disfluencies. In some of our recent work we’re showing that treating talk as sanitised text can adversely impact voice user interfaces. However, this is still a minority position. Googlers Dan Walker and Dan Liebling represent the mainstream view well in this blog post:  Fair enough, you might say.

Sawing off the branch you’re sitting on

One of the benefits of today’s preprint culture is that it is possible to provide constructive critique of pending work before it is out, thereby enabling a rapid cycle of revision before things are committed to print. I have myself benefited from comments on preprints, and have acknowledged such public pre-publication reviews in several of my papers. The below remarks are shared in that spirit.

Pitfalls of fossil-thinking I

A preprint claims that “ideas from theoretical linguistics have played no role in [NLP]”. Outside the confines of Chomskyan linguistics folks have long been working on  incorporating storage, retrieval, gating and attention in theories of language, with direct relevance to computational models. The only way to give any content to the claim is by giving the notion “theoretical linguistics” the narrowest conceivable reading.

This Lingbuzz preprint by Baroni is a nice read if you’re interested in linguistically oriented deep net analysis. I did feel it’s a bit hampered by the near-exclusive equation of linguistic theory with generative/Chomskyan aps.

Linguistic roots of connectionism

Over two years ago I wrote about the unstoppable tide of uninformation that follows the rise of large language models. With ChatGPT and other models bringing large-scale text generation to the masses, I want to register a dystopian prediction.

Monetizing uninformation: a prediction

I have always had a fondness for things considered marginal in linguistics. The tide may be turning for at least some marginalia: work on ideophones is clearly on the rise, and initiatives such as Martina Wiltschko’s Eh lab at UBC and a new nonlexical vocalizations project at Linköping University show there is significant interest in this area.

Composite ideophone-gesture utterances in the Ashéninka Perené ‘community of practice’, an Amazonian Arawak society from Central-Eastern Peru

New!

Some of this is now published here (open access, free for all!): Dingemanse, Mark. 2020.

What is ‘non-lexical’? Notes on non-lexical vocalisations, II

The oldest written fragments of Siwu found so far come from Rudolph Plehn (1898).

 1

Besides some words and phrases (edited and published in 1899 by his friend Seidel), Plehn took down two lines of songs. To one of them I devoted a post some time ago.

Bingo! Refinding the oldest specimen of Siwu

I have always had a fondness for things considered marginal in linguistics. The tide may be turning for at least some marginalia: work on ideophones is clearly on the rise, and initiatives such as Martina Wiltschko’s Eh lab at UBC and a new nonlexical vocalizations project at Linköping University show there is significant interest in this area. Part I from my notes on a workshop on ‘Ideophones and non-lexical vocalizations’.

Making new ideophones in Siwu

On the margins of language: Ideophones, interjections and dependencies in linguistic theory

Bodily Quoting in Dance Correction

Turn organization and bodily-vocal demonstrations

Last week I was happy to present my work at a workshop on

 Ideophones and nonlexical vocalisations

in Linköping, Sweden, organised by Leelo Keevallik and Emily Hofstetter. This was the kick-off for a new project on “Non-lexical vocalisations”. It was my first time in Linköping and it was great getting to know the vibrant community of interaction researchers from across departments.

A variety of vocal depictions: Notes on non-lexical vocalisations, I

One of the key tasks scientists need to master is how to manage bibliographic information: collecting relevant literature, building a digital library, and handling citations and bibliographies during writing. This tutorial introduces Zotero (www.zotero.org), an easy to use reference management tool made by scholars for scholars. The tutorial covers the basics of using Zotero for collecting, organizing, citing and sharing research.

Slides for a hands-on Zotero workshop

We don’t generally see PhD dissertations as an exciting genre to read, and that is wholly our loss. As the publishing landscape of academia is fast being homogenised, the thesis is one of the last places where we have a chance to see the unalloyed brilliance of up and coming researchers. Let me show you using three examples of remarkable theses I have come across in the past years.

Mindblowing dissertations

Writing is thinking. The writing process is the most neglected part of our job. We spend millions on fancy equipment and uncountable hours on training for using this or that toolkit. Yet we assume the BA-level academic writing course we once followed is sufficient; the rest we’ll just learn on the job and hopefully soon we’ll automate away with LLMs. It is all formulaic anyway.

Like anyone active in the space of LLMs and generative AI, I get a growing amount of invitations to speak at conferences and colloquia on something or other LLM-related. A subset of these invites is about the relation between LLMs and academic writing. Recently I had the occasion to respond to such an invite.

Find your own voice: against LLM slop in academic writing

It’s easy to forget amidst a rising tide of synthetic text, but language is not actually about strings of words, and language scientists would do well not to chain themselves to models that presume so. For apt and timely commentary we turn to Bronislaw Malinowski who wrote:  In follow-up work, Malinowski has critiqued the unexamined use of decontextualised strings of words as a proxy for Meaning:  Malinowski did not write this on his substack,

Malinowski (1922) on Large Language Models

Featured philologist of today is Joh. Bernhard Schlegel, for providing us with precious data on ideophones (expressives) in nineteenth-century Ewe, a Kwa language of southeastern Ghana.

This is the first post in a series. Featured philologist of today is Joh. Bernhard Schlegel, for providing us with precious data on ideophones (expressives) in nineteenth-century Ewe, a Kwa language of southeastern Ghana. But since this is the first post on ideophones here, let’s first try to answer the obvious question: what are ideophones, anyway?

Early sources on African ideophones, part I: Schlegel on Ewe, 1857

Having been a small and quite isolated language for centuries, Siwu was relatively late to attract attention from outsiders. Europeans in search for gold to buy and people to enslave for the most part stayed near the coast.

‘Remnants of some ancient tribal idiom’: deciphering the oldest Siwu to appear in print

One of my projects here at The Ideophone has been to track down early sources on ideophonic phenomena. For example, I have suggested that we may call the 1850’s the decade of the discovery of ideophones in African linguistics. But we can push back the linguistic discovery of ideophones a little further by looking to other traditions.

Aduerbia sonus: Ideophones in two 17th century grammars of Japanese

This is the first ever published account of a visit to Akpafu. It was written down by David Asante, a Twi pastor who travelled throughout today’s Volta Region in the company of some white missionaries. The journey took place in January 1887; the date of the visit to Akpafu was January 25th, 1887.

A visit to Akpafu by David Asante, 1887

Words evolve not as blobs of ink on paper but in face to face interaction. The nature of language as fundamentally interactive and multimodal is shown by the study of ideophones, vivid sensory words that thrive in conversations around the world. The ways in which these “Lautbilder” enable precise communication about sensory knowledge has now for the first time been studied in detail.

Words   Few professions should be more familiar with the nature of words than academia. Words are the currency of our trade. They record our cumulative progress and they measure our productivity as we disperse our ideas through articles and books. How easy is it to fall in love with the printed word, black symbols on a white page, tidy spaces separating units of thought like stars dotting the skies of conceptual clarity!

How to paint with language

Will synthetic text generators usher in a new age of creative thinking? The remarkable fluency of large language models may make them interesting tools for rapidly exploring semantic and stylistic spaces, yet the deceptive ease with which they generate output also provides countless new ways of appropriating ideas and erasing authorship.

Human creativity in the age of text generators

Funeral dirges (

 sìnɔ

in Siwu) are sung during the period of public mourning preceding a burial. The musical structures of these dirges, the performances, and their place in the larger context of the funeral have been described in some detail by Agawu (1988) and before him by the German missionary Friedrich Kruse (1911); however, the linguistic aspects of the genre have not received any attention so far.

I thought I had company (a Mawu dirge)

Google Scholar is useful, but its inclusiveness and mix of automatically updated and hand-curated profiles means you should never take any of its numbers at face value. Case in point: the power couple Prof.

Summary:
 
 Google Scholar is useful, but its inclusiveness and mix of automatically updated and hand-curated profiles means you should never take any of its numbers at face value. Case in point: the power couple Prof.

Some things you need to know about Google Scholar

The Ideophone

Large language models make it entirely trivial to generate endless
amounts of seemingly plausible text. There's no need to be cynical to
see the virtual inevitability of unending waves of algorithmically tuned
AI-generated uninformation: the market forces are in place and they will
be relentless.

I say **uninformation** against the backdrop of Bateson's
tongue-in-cheek definition of information as 'a difference that makes a
difference'. If we don't know (or can't tell) the difference anymore, we
are literally un-informed.

It is likely that a company like OpenAI sees some of this and that
they're keeping, for instance, time-stamped samples of AI-hallucinated
content to enable some degree of textual provenance --- but given how
hard it is to deal with content farms already I think there's little
reason to be optimistic.

Which has an important consequence. The web makes up a large chunk of
the data feeding GPT3 and kin. Posting the output of large language
models online builds a feedback loop that cannot improve quality (unless
we have mechanisms for textual provenance) and so will lead to
uninformation feeding on uninformation.

All ingredients for an information heat death are on hand. True
human-generated and human-curated information ---of the kind produced,
for instance, by academics in painstaking observations and
publications--- will become more scarce, and therefore more valuable.
Counterintuitively, there was never a better time to be a scholar.

- Bateson, G. (1979). *Mind and Nature*. New York: E.P. Dutton.
