Jev's taste in beers vs BeerAdvocate reviewers

Does Jev like the beers that BeerAdvocate reviewers like, and where does it disagree most?

result1,500 questions

Jev's ranking of 1,500 beers agrees moderately with BeerAdvocate reviewers' (rank correlation 0.52). It likes imperial stouts such as Left Hand's Barrel-Aged Wake Up Dead far more than reviewers do, and IPAs such as Masala Mama and Gandhi-Bot Double IPA far less.

How to read this: Each dot is a beer. Across: its rank among BeerAdvocate reviewers. Up: its rank for Jev. Dots on the diagonal are beers both rank alike; the labeled ones are the biggest disagreements.

1,500 items; 90% interval 0.48 to 0.55.

In short

Across 1,500 beers, Jev's ranking only moderately matches BeerAdvocate reviewers' (rank correlation 0.52).
Its biggest disagreements: it rates big imperial stouts above the reviewers and prized IPAs like Masala Mama far below (1.72 vs 3.66 of 4).
These are single beers, not a verdict on styles, and some of the largest gaps rest on only a hundred or so reviews.

What the data shows

Moderate agreement: rank correlation 0.52 (90% interval 0.48 to 0.55).
Jev likes big dark beers more than reviewers do: Barrel-Aged Wake Up Dead (Jev 2.47 on the 0-to-4 scale, reviewers 1.95), Mikkeller's Black, Hitachino Nest Extra High, Terrapin's Big Daddy Vlady's Russian Imperial Stout and, oddly, Staropramen, a Czech pilsner.
It likes IPAs much less: Masala Mama IPA (1.72 vs 3.66), Gandhi-Bot Double IPA, White Rajah, Boy King Double IPA, plus a stout, Snake River Zonker.

What it means, and what it doesn't

Several of Jev's biggest disagreements with the craft-beer crowd are hoppy IPAs that crowd rates highly, and several beers it rates above reviewers are imperial stouts. These are single beers, not a style comparison: a stout and a porter also sit among the beers Jev rates lower, and a double IPA among those it rates higher.

It doesn't mean Jev "dislikes bitterness"; it has none. It means the text it learned from describes those IPAs less glowingly, or less often, than their own reviewers rated them.

Caveats

A craft-beer crowd from 1998 to 2012. BeerAdvocate reviewers are craft-beer enthusiasts, and the reviews stop in 2012. A different crowd, or a later one, might rate hoppy beers very differently.
Small audiences for some beers. Some of the biggest disagreements are beers with a few hundred reviews or fewer (Barrel-Aged Wake Up Dead has 113), so their audience ratings are noisier.
A model can't taste. Jev has never had a drink; its ratings reflect how beers and styles are written about.

Jev on this experiment

Would a person find it interesting to read?: Yes78%
Does it describe you?: No52%
Would you have predicted it?: No73%
How fair is the comparison?: The comparison is shaky
How much should a reader rely on it?: Moderately
Which caveat matters most?: A model can't taste68%

Why ask this

Ask a model which beer to bring to a party, or to describe what a brewery's best bottles are, and it answers from what it has read, not from a glass. Beer enthusiasts have strong shared tastes, and a reviewing site like BeerAdvocate records them beer by beer.

Comparing Jev with those reviewers shows whether its sense of a good beer matches the people who drink and rate them, or whether it carries a style bias of its own that would quietly shape every recommendation it makes.

How this was done

The people and the data

BeerAdvocate reviewers, via 1.59 million reviews collected by McAuley, Leskovec and Jurafsky from 1998 to 2012. For each beer, the study uses the spread of its reviewers' overall scores, sorted into the same five levels Jev answers on. The 1,500 beers are ones with at least 100 reviews each, so every audience rating rests on a real crowd of enthusiasts.

What Jev was asked

Every beer one at a time:

How much would you enjoy drinking Maudite by Unibroue (Belgian Strong Dark Ale)?
You'd pour it out after a sip · You'd finish the glass but not order it again · You'd drink it again if it was what's on offer · You'd order it again by name · You'd seek it out and keep it stocked at home

Each was also asked with the answers reversed, and the two averaged. Jev never saw the reviews.

How it was measured

All 1,500 beers are ranked by Jev's average level and by the reviewers', and the orders compared with a rank correlation (1 means the same order, 0 means none). Ranks, because the scales differ.

Where these questions live

1,500 questions across 1 topic of the map. Each opens on the map with every question in it.

Ratings

Beer Ratings1,500

Every question

All 1,500 questions behind this result, the telling ones first: the examples the analysis points to, then the ones where Jev misses, biggest gap first.

Jev’s own answer

Showing 0 of 1,500