Atlas › Reading the crowd

Case study 71 of 198

Would you rather: Jev vs 1.5 million votes each

On 750 would-you-rather questions voted on by millions (either.io), does Jev pick what most people pick, and where does it split from them hardest?

result750 questions

Jev sides with the majority on 64% of 750 would-you-rather questions (69% where voters were clear). Its sharpest split: it would rather ride in a hot air balloon than a hovercraft (90%), where 80% of 1.4 million voters chose the hovercraft. Next: eat 50 beetles rather than 100 ants (87%), where 82% of 1.2 million voters chose the ants.

0%25%50%75%100%0%25%50%75%100%Ride in a hot air balloon /…Eat 100 ants / Eat 50 beetl…Check your email first ever…voters choosing AJev choosing A

How to read this: Each dot is a question. Across: the share of voters who chose the first option. Up: Jev's probability for it. The labeled dots are the biggest disagreements, where Jev and a million voters went opposite ways.

750 questions, median 1,271,824 votes each; rank correlation 0.40. Jev's guess of most people matches the majority 66% of the time.

In short

  • On 750 dilemmas, Jev picks the side most voters picked about two times in three (64%).
  • Its biggest splits favor the richer-sounding experience, a balloon over a hovercraft and books over songs, where over 80% of a million-plus voters disagreed.
  • Asked what most people would pick, it mostly names its own choice, so it predicts the crowd no better (66%).

What the data shows

reading the crowd
Left Exit 12 Off Ramp meme: hovercraft (80% of voters); hot air balloon; Jevhovercraft (80% of voters)hot air balloonJev
How funny is this meme? Jev: 3/5, funny11%234%360%45%50%
  • Majority matched 64% of the time, and 69% where voters were clear. What it thinks most people would pick matches about as often (66%). Across all questions, Jev's probabilities follow the vote shares only loosely (rank correlation 0.40).
  • The balloon. 80% of 1.4 million voters would rather ride in a hovercraft; Jev picks the hot air balloon (90%), and thinks most people would too.
  • The bugs. 82% of 1.2 million voters would rather eat 100 ants than 50 beetles; Jev picks the beetles (87%).
  • The ghost. 85% would rather be stalked by a ghost for life than by a demon for three days and then die; Jev picks the demon.
  • Books vs songs. 82% would rather never read another book than never hear another song; Jev picks the opposite.

What it means, and what it doesn't

On two-thirds of trivial preferences, Jev lands where the crowd does. Reading the biggest misses (a handful, picked by size, not a measured pattern), Jev tends to pick the experience that sounds richer or more meaningful (the balloon, books over songs) and the smaller number (50 beetles), where voters go with gut feel (hovercraft, ants, the ghost you can live with).

It doesn't mean Jev is wrong, since there's no wrong answer. It means its taste on these is its own, and a model asked to predict what people would pick shouldn't assume they'd pick what it would.

Caveats

  • A game site's voters. The votes come from either.io, a would-you-rather game site. Voters are self-selected, anonymous, and often voting for fun; the numbers are enormous but the crowd is not a sample of anyone in particular.
  • Only popular questions. Questions were kept with at least 1,000 votes (the median question has over a million), which are the site's most-played ones. 1,000 were asked; a quarter were hidden by the content filter (sexual and other sensitive items), leaving 750.
  • A copy of the site's data. The data is a public scrape of either.io published on Kaggle, and it is used for private research only.

Jev on this experiment

Would a person find it interesting to read?
Yes78%
Does it describe you?
No64%
Would you have predicted it?
No62%
How fair is the comparison?
The comparison is reasonable
How much should a reader rely on it?
Moderately
Which caveat matters most?
A game site's voters78%

Why ask this

Would-you-rather questions are pure preference: there's no right answer, only what people would pick. The site either.io has collected votes on thousands of them, often more than a million votes per question, which makes the crowd's answer about as stable as a preference ever gets.

Where Jev agrees with a million people, it has absorbed ordinary taste. Where it confidently disagrees, the disagreement is a small, plain look at its quirks.

How this was done

The people and the data

Would-you-rather questions from either.io, a game site where visitors vote on one dilemma after another, taken from a public scrape of the site published on Kaggle. About 1,140 questions have at least 1,000 votes; 1,000 of them were asked, and 750 remain after a content filter hid sexual and other sensitive items. The median question has 1,271,824 votes, so each crowd share is about as stable as a preference gets.

What Jev was asked

Each dilemma exactly as the site words its two options:

Which would you rather?

Ride in a hot air balloon · Ride in a hovercraft

Jev answered for itself, and again for what it thinks most people would say.

How it was measured

How often Jev's own pick matches the majority, and the same for what it thinks most people would pick. Then the questions where Jev's probability is furthest from the vote share, among questions where voters were clear (60% or more one way).

Where these questions live

750 questions across 5 topics of the map. Each opens on the map with every question in it.

Every question

All 750 questions behind this result, the telling ones first: the examples the analysis points to, then the ones where Jev misses, biggest gap first.

Jev’s own answer
    Showing 0 of 750