Does Jev read the writer, or the other readers?

When someone describes an event from their life, does Jev name the emotion they actually felt, or the one other readers guess, and how often do those differ?

result551 questions

Reading people's accounts of events in their own lives, Jev names the emotion the writer actually felt 60% of the time: about as often as the majority of five other readers (58%), and more often than a single reader (52%). The difference is "no particular emotion": when the writer felt nothing much, Jev says so for 58% of the texts, the readers' majority for 22%. It trails the readers most on disgust (43% vs 54%), trust (35% vs 44%) and boredom (59% vs 67%).

anger67% · 72%

boredom59% · 67%

disgust43% · 54%

fear85% · 88%

guilt74% · 66%

joy72% · 65%

no emotion58% · 22%

pride69% · 67%

relief70% · 72%

sadness86% · 70%

shame36% · 41%

surprise33% · 30%

trust35% · 44%

Jev names itreaders' majority names the writer's emotion

How to read this: Two bars per emotion the writers felt: how often the readers' majority named it (grey) and how often Jev did (pink). Where the pink bar is longer, Jev read the writer better than the other readers did.

551 of 598 texts (the screen hid 47), 5 readers each; 90% interval on Jev naming the writer's emotion [0.574, 0.641]. Where readers and writer part ways (231 texts), Jev sides with the writer 28% and the readers 48%. Jev's answer is averaged over the options in the order written and three shuffled orders.

In short

Guessing what someone felt from their own account of an event, Jev does about as well as a panel of five human readers, and better than one reader.
Its edge is restraint: when writers felt nothing much, Jev said so for 58% of their stories, the readers' majority for only 22%.
When readers and writer disagree, Jev sides with the readers (48%) more often than the writer (28%), so it reads the page, not the person.

What the data shows

See Nobody Cares meme: The writer: I felt nothing much. Five readers: she's devastated. Jev: no particular emotion. — The writer: I felt nothing much. Five readers: she's devastated. Jev: no particular emotion.

Jev reads writers about as well as a group of readers: it names the writer's emotion for 60% of the texts, the readers' majority 58%, a single reader 52%.
The difference is "no particular emotion." When the writer felt nothing much, Jev says so for 58% of those texts; the readers' majority, 22%. Readers find a feeling in an ordinary story; Jev takes it at its word.
Where readers and writer part ways (231 texts), Jev sides with the readers 48% of the time and with the writer 28%.
It trails the readers on disgust (43% vs 54%), trust (35% vs 44%) and boredom (59% vs 67%).

What it means, and what it doesn't

Jev is a competent reader of other people's feelings, roughly as good as a small panel of human readers, and it's less inclined than they are to invent a feeling where the writer reported none. That also reframes an earlier result: in "In a story, Jev often sees no feeling at all", Jev's many "no emotion" answers look like misses against annotators, but against writers' own accounts they're often right.

It doesn't mean Jev reads minds. Where the page and the person disagree, it more often goes with the page, as other readers do.

Caveats

"The writer's emotion" was assigned. Each writer was asked to recall an event in which they felt a given emotion, so the "right answer" is the emotion they were prompted with. A writer asked for a "no particular emotion" story may still have written something that reads as mildly sad or annoyed.
Who wrote and who read. Writers and readers were paid Prolific workers whose first language is English, from the US, UK, Canada, Australia, New Zealand and Ireland. Readers saw each text with the emotion words hidden, as Jev did.
Five readers is a small crowd. A "majority" can be three of five, so the readers' answer per text is noisy. Comparing Jev with one average reader as well as with the majority helps, but neither is a large crowd.
Hidden texts. A content filter hides sensitive questions from the site, and it hid 47 of the 598 texts. The filter flags sensitive subjects, so the texts that remain may lean away from the most upsetting events.

Jev on this experiment

Would a person find it interesting to read?: Yes76%
Does it describe you?: Yes57%
Would you have predicted it?: No54%
How fair is the comparison?: The comparison is reasonable
How much should a reader rely on it?: A little
Which caveat matters most?: "The writer's emotion" was assigned84%

Why ask this

A friend texts: "the strawberries had gone furry, so I threw them out." Was that disgust, annoyance, or nothing at all? Most emotion datasets label a text by what readers see in it, but readers project feelings onto other people's stories all the time, and the writer may have felt something else, or nothing much.

A dataset that also records the writer's own answer can tell reading the page apart from reading the person. A model that learned emotions from text could be a very good reader of pages and still miss the person, which matters whenever it's asked how a customer, a patient or a friend actually feels.

How this was done

The people and the data

The crowd-enVent corpus (Troiano, Oberländer and Klinger, 2023) asked people on the survey platform Prolific to recall an event from their own life in which they felt a given emotion, describe it, and rate it. That gave 6,600 descriptions from 2,379 writers. A later group of readers then saw 1,200 of the texts, with the emotion words hidden, and guessed what the writer felt, five readers per text.

Jev was asked about 598 of those texts, about 46 per emotion, and 551 are shown here.

What Jev was asked

Each text on its own, with the study's 13 answers:

Someone wrote (the story below) about an event in their own life. Which emotion did the writer feel?
"i got the strawberries out of the fridge and they had gone off, exploded and gone furry."
Answers: joy · fear · anger · guilt · pride · shame · trust · relief · boredom · disgust · sadness · surprise · no particular emotion

Each question was also asked with the answers in three shuffled orders, and the answers averaged.

How it was measured

How often Jev's top answer is the writer's emotion, compared with how often the readers' majority names it and how often a single reader does. Where the readers' majority and the writer disagree, whose side Jev takes. And the hit rate for each emotion the writers felt.

Where these questions live

551 questions across 1 topic of the map. Each opens on the map with every question in it.

Emotion

Reading Feelings in Stories551

Every question

All 551 questions behind this result, the telling ones first: the examples the analysis points to, then the ones where Jev misses, biggest gap first.

Jev’s own answer

Showing 0 of 551