Jev plays Family Feud like a quiz

Given the answers a Family Feud survey got ('Name something a knight needs for a jousting match'), does Jev pick the one most people said first?

result146 questions

Jev picks the survey's top answer on 47% of 146 Family Feud questions, against 19% by chance. Its biggest misses are sensible answers few people gave: "Name a measurement people know on their body" got Height from Jev and Waist from 65% of people; "Name something a knight needs for a jousting match" got Lance from Jev and Horse from 55% of people.

1. 1st47%

2. 2nd20%

3. 3rd18%

4. 4th9%

5. 5th3%

6. 6th4%

chance: 19%

How to read this: Each bar is how often Jev's pick was the survey's number one answer, its number two, and so on down to the sixth; the dotted line marks chance, where a random pick would land on number one about 19% of the time.

146 questions; 90% interval [0.397, 0.534]; Jev's pick is in the survey's top two 66% of the time.

In short

Jev names the survey's favorite on nearly half of 146 Family Feud questions, and puts it in the top two 66% of the time.
Its misses pick the defining answer over the first one people think of, like Paris as a place people hope to go, where 58% said Heaven.
Predicting what people will say and knowing the "right" answer are different skills, and Jev leans toward the second.

What the data shows

Oh yeah! Oh no... meme: Jev: "height!"; survey says: waist (65%)

Number one answer on 47% of questions, against 19% by chance; in the survey's top two 66% of the time.
Some of the biggest misses are defensible (examples hand-picked). - "Name a measurement people know on their body": 65% of people said waist; Jev said height. - "Name an occupation whose members wear business suits": 61% said lawyers; Jev said banker. - "Name something a knight needs for a jousting match": 55% said horse; Jev said lance.

What it means, and what it doesn't

In these examples Jev plays Family Feud like a quiz: it names the most defining or "correct" answer (a lance is what makes a joust a joust), while people name the first thing they picture (the horse). Its guesses of the crowd's top answer are right far more often than chance, but it underweights the obvious.

It doesn't mean Jev is wrong about knights. It means that predicting what people will say and knowing the answer are different, and Jev leans toward the second.

Caveats

A game-show survey. The answers come from the TV show's own surveys of about 100 people each, scraped from fan sites by the dataset's authors. They reflect the show's American audience and the moment each survey was run.
Clusters grouped by the researchers. Similar answers ("lawyer", "lawyers", "attorney") were grouped into one option by the dataset's authors, and Jev chose among the top four to six groups. How they were grouped affects which answer counts as "first".
Small set. 146 questions give a clear overall rate, but not enough to say which kinds of questions Jev reads better.

Jev on this experiment

Would a person find it interesting to read?: Yes80%
Does it describe you?: No53%
Would you have predicted it?: No64%
How fair is the comparison?: The comparison is reasonable
How much should a reader rely on it?: Moderately
Which caveat matters most?: Clusters grouped by the researchers82%

Why ask this

Family Feud doesn't reward the best answer; it rewards the most common one. "Name something a knight needs for a jousting match": the right answer is whatever most of a hundred surveyed people blurted out first.

That makes it a neat test of a different skill from knowledge. Knowing the correct answer is one thing; knowing what ordinary people think of first is another, and it's the skill a model needs to predict what people will say.

How this was done

The people and the data

ProtoQA (Boratko and colleagues, 2020) collected Family Feud survey questions with their answer counts, scraped from fan sites that record the show's boards. Each survey asked about 100 people, and the researchers merged answers that mean the same thing ("lawyer", "lawyers", "attorney") into one group, so every question comes with a ranked list of groups and the share of people behind each. The experiment uses 146 questions with at least four answer groups.

What Jev was asked

Each question with the survey's top answers as options:

Which of these would most people name first when asked: "Name a measurement people know on their body."
Waist · Height · Weight · Shoe size

How it was measured

How often Jev's top pick is the survey's number one answer, compared with a random pick (one over the number of options, 19% on average). The analysis also looks at where Jev's pick ranked in the survey, and at the biggest misses, where the survey's favorite was far ahead.

Where these questions live

146 questions across 44 topics of the map; the 16 biggest are shown. Each opens on the map with every question in it.

Every question

All 146 questions behind this result, the telling ones first: the examples the analysis points to, then the ones where Jev misses, biggest gap first.

Jev’s own answer

Showing 0 of 146