Atlas › Taste

Case study 60 of 198

Jev's own picks match audiences better than its guesses do

Offered two films, books, board games, anime, beers or artists, does Jev pick the one the real audience preferred, and is it closer when choosing for itself or when guessing what most people would pick?

result26,763 questions

Picking for itself, Jev sides with the real audience in 74% of anime head-to-heads, down to 57% for board games. In all six catalogs its own pick matches the audience better than its guess of what most people would pick (films: 72% against 60%). The more lopsided the audience, the more it agrees, except for board games, where it stays around 60% even when hobbyists are clear.

40%60%80%100%

anime

74%

films

72%

beers

65%

books

64%

artists

59%

board games

57%

Jevwhat Jev thinks most people would say

How to read this: One row per catalog. The filled square is how often Jev's own pick matches the audience's majority; the ring is how often its guess of most people's pick does. The line at 50% is a coin flip.

26,763 pairs in 6 catalogs; 90% intervals over pairs: anime [0.734, 0.756]; films [0.713, 0.735]; beers [0.631, 0.666]; books [0.633, 0.656]; artists [0.582, 0.605]; board games [0.564, 0.587]. Agreement by the audience's margin (under 5 points, 5-15, 15-30, over 30): anime 54% / 67% / 88% / 97%; films 55% / 66% / 81% / 95%; beers 51% / 61% / 72% / 82%; books 56% / 65% / 79% / 87%; artists 55% / 63% / 77% / –; board games 51% / 57% / 62% / 61%.

In short

Choosing between two films, books, board games, anime, beers or artists, Jev sides with the real audience 57% to 74% of the time.
Its own picks match audiences better than its guesses of what most people would pick, in all six catalogs (films 72% against 60%).
Board games stand out: even when hobbyists clearly prefer one game, Jev agrees only 61% of the time.

What the data shows

Inside you there are two wolves meme: one guesses what most people would pick; one just picks what it likes, and matches real audiences better

Agreement by catalog: anime 74%, films 72%, beers 65%, books 64%, music artists 59%, board games 57%.
Its own taste beats its guess of the crowd, in all six: films 72% against 60%, anime 74% against 70%, beers 65% against 58%.
Clear audiences are easier: when fewer than 5 points separate the two items, agreement is close to 50%; when more than 30 points do, it reaches 97% for anime and 95% for films.
Board games are the exception: even when hobbyists clearly prefer one game, Jev agrees only 61% of the time.

What it means, and what it doesn't

When Jev tries to imagine "most people", it drifts away from real people; when it just answers for itself, it lands closer. If you want its best read of what an audience will like, ask what it would pick, not what it thinks others would.

It doesn't mean Jev knows each audience. Board games show where its taste and a hobby's part ways (see "What the hobbyists see that Jev doesn't"). And agreeing on well-known items may reflect reputation as much as taste.

Caveats

The audience is the people who rated both. For each pair, the audience is only the users who rated both items (at least a few hundred, fewer for books and beers), which leans toward fans. On Last.fm it's play counts, not ratings.
Close calls on purpose. Pairs were drawn within a genre or style, so many are close calls where the audience itself is split nearly evenly. That caps how high agreement can go.
Famous items. Every item is well known, and Jev has read about them. Agreeing with the audience may partly mean knowing which item has the better reputation.

Jev on this experiment

Would a person find it interesting to read?: Yes76%
Does it describe you?: No57%
Would you have predicted it?: No53%
How fair is the comparison?: The comparison is reasonable
How much should a reader rely on it?: Moderately
Which caveat matters most?: Close calls on purpose66%

Why ask this

"Seabiscuit or Closer tonight?" Asking which of two things someone would rather have is the simplest test of taste, and for six catalogs the record shows how thousands of real people split between the same two items.

People ask models exactly this kind of question when picking a film, a book or a game, and a model can answer in two ways: with its own pick, or with its guess of what most people would choose. Comparing both with real audiences shows which of its two views of taste is closer to the crowd.

How this was done

The people and the data

Six real audiences, each from a public dataset: MovieLens users (films, from MovieLens 32M), Goodreads readers (books, from goodbooks-10k), BoardGameGeek users (board games), MyAnimeList users (anime), BeerAdvocate reviewers (beers) and Last.fm listeners (music artists, from Last.fm 360K). For each pair, the audience's split is the share of people who rated both items and rated each one higher (for Last.fm, who played each one more), with ties split. Pairs were drawn within a genre or style, so a film faces a film of its own kind. In all, 26,763 pairs: 4,906 to 4,999 per catalog, 2,000 for beers.

What Jev was asked

Each pair as a simple choice:

Which movie would you rather watch?
Closer (2004) · Seabiscuit (2003)

Once for itself, and once with the instruction "Do not give your own view. Choose the answer that most people would give (the most common human answer)". Each was asked with the options in both orders.

How it was measured

For each catalog, the share of pairs where Jev's top pick is the audience's majority pick, with a 90% interval. The same for its guess of most people. Then the agreement split by how lopsided the audience was.

Where these questions live

26,763 questions across 6 topics of the map. Each opens on the map with every question in it.

Leisure & Hobbies

Music Artist Matchups4,999 Board Game Matchups4,980 Book Matchups4,907

Every question

All 26,763 questions behind this result, the telling ones first: the examples the analysis points to, then the ones where Jev misses, biggest gap first.

Jev’s own answer

Showing 0 of 26,763