Jev hears and smells words like people but underrates seeing them

Asked how much it experiences each word through sight, hearing, touch, taste and smell, does Jev give the sensory profile people give?

result11,029 questions

Jev's sense of how much things are smelled and tasted matches people's (1.0 vs 0.9 and 0.8 vs 0.8 on a 0-5 scale), but it rates how much they are seen far lower than people do: 2.1 where people say 3.9. Sight is also the sense where its ranking of words agrees least with people's (0.49, against 0.74 for hearing). For words people experience mainly by sight, it names sight 75% of the time and touch 13%.

Sight2.1 · 3.9

Hearing1.4 · 1.5

Touch1.5 · 2.0

Taste0.8 · 0.8

Smell1.0 · 0.9

Jev (0-5)people (0-5)

How to read this: One pair of bars per sense, on 0 to 5: how strongly Jev says words are experienced through that sense (magenta) and how strongly people do (grey). The numbers on each row are Jev's average, then people's.

9,898 sense ratings over about 1,980 words per sense; 1,131 dominant-sense questions. With the levels reversed, the sight gap stays (the robust level already averages both orders).

In short

Jev matches people on how much words are smelled, tasted and heard, but rates sight far lower: 2.1 of 5 where people say 3.9.
Asked to pick one sense, it still names sight for 75% of words people mainly know by seeing, so the gap is in degree, not in which sense.
One likely reason is that text rarely spells out what everyday things look like; part may also come from how the answer levels written for this project describe seeing.

What the data shows

Are they in the room with us right now? meme: Jev on how much we experience words by seeing them: 2.1 of 5 (people 3.9) — Jev on how much we experience words by seeing them: 2.1 of 5 (people 3.9)

Smell and taste: close. 1.0 vs 0.9 for smell, 0.8 vs 0.8 for taste. Hearing: 1.4 vs 1.5, and the best agreement in ranking (0.74).
Touch: a little low. 1.5 vs 2.0.
Sight: far too low. 2.1 vs 3.9. People say they experience most concrete words strongly through sight; Jev says moderately at most. Its ranking of words by sight is also the least like people's (0.49).
But it knows sight dominates when asked directly: for words people experience mainly by sight, Jev names sight 75% of the time, touch 13%.

What it means, and what it doesn't

Jev knows that sight is the main sense for most objects when asked to choose, but it underrates how much seeing is involved when asked about each word on its own. One likely reason is text itself: people write about how things smell, taste and sound when it's noteworthy, and rarely spell out what everyday things look like, because everyone can see them.

It isn't evidence that Jev "can't see". It's a gap in how strongly its text-based picture weights the visual, and part of it may come from how the project's answer levels describe seeing.

Caveats

A mapped scale. People rated 0 to 5; Jev chose among five described levels written for this project. Jev's levels are mapped onto 0 to 5 evenly. The sight gap is too large to be a mapping artifact, and the rankings, which don't use the mapping, point the same way.
"You" is a model. The question asks "how much do you experience" the word. Jev has no senses; it may be answering how much the word's meaning is about seeing, which is a different question from how people experience it.
The level wording. The levels describe sight as "one of the main ways I experience it" or "what it is mostly about". Jev may reserve those for words that are about seeing (colors, views) rather than things people mostly know by looking at them.
Averages only. The norms publish averages per word, not each rater's answer, so the comparison uses averages.

Jev on this experiment

Would a person find it interesting to read?: Yes73%
Does it describe you?: Yes58%
Would you have predicted it?: Yes68%
How fair is the comparison?: The comparison is shaky
How much should a reader rely on it?: Moderately
Which caveat matters most?: "You" is a model86%

Why ask this

How do you know a "mustache"? Mostly by seeing it. "Thunder"? By hearing. "Velvet"? By touch. The Lancaster Sensorimotor Norms asked people how much they experience tens of thousands of words through each sense, and the answers describe how grounded words are in the body: for most concrete words, sight dominates.

A model has no senses. It knows words from text, where the look of things is rarely described because everyone can see it. Which sense it misjudges most is a clue to what text leaves out.

How this was done

The people and the data

The Lancaster Sensorimotor Norms (Lynott and colleagues, 2020) cover 39,707 English words, rated by online participants in the US and UK, recruited through Mechanical Turk and Prolific, on how much they experience each word through sight, hearing, touch, taste and smell, on a scale from 0 (not at all) to 5 (greatly). The study uses about 1,980 concrete, common words per sense. A second set of questions uses the same norms to ask which single sense dominates a word, where one clearly does.

What Jev was asked

One question per word and sense, with five described answers written for this project:

How much do you experience "mustache" by seeing?
Not at all: seeing plays no part in experiencing it · Slightly: its sight comes up only now and then, as a minor detail · Moderately: its sight is one noticeable part of experiencing it, alongside others · Strongly: its sight is one of the main ways I experience it · Greatly: seeing is central to experiencing it; its sight is what it is mostly about

Plus 1,131 questions like "Through which sense do you mostly experience ...?", with sight, hearing, touch, taste, smell and body sensation as the options. Each was also asked with the answers reversed or shuffled, and averaged.

How it was measured

For each sense, Jev's average rating on the 0 to 5 scale next to people's, and how closely the two rankings of words agree (a rank correlation: 1 means the same order). For the dominant-sense questions, how often Jev names the sense the norms say dominates.

Where these questions live

11,029 questions across 257 topics of the map; the 16 biggest are shown. Each opens on the map with every question in it.

Every question

All 11,029 questions behind this result, the telling ones first: the examples the analysis points to, then the ones where Jev misses, biggest gap first.

Jev’s own answer

Showing 0 of 11,029