Does Jev know what things cost in 1985?

Asked what an item cost in US cities in 1985, 1995, 2005 and 2015, does Jev know the old prices as well as recent ones, and which way does it err?

result126 questions

Asked what everyday things cost in past years, Jev picks the right price range 22% to 33% of the time for 1985, 1995 and 2005, but only 3% of the time for 2015. Its misses lean high for 1985 (+0.9 ranges) and low for 2015 (-2.2): it remembers the past as pricier, and the recent past as cheaper, than it was.

0%25%50%75%100%

1985

+0.9 bins

1995

+0.0 bins

2005

-0.9 bins

2015

-2.2 bins

Jev

How to read this: One row per year. The magenta square is the share of items where Jev picked the right price range; the label at the right is the average lean of its misses, in price ranges ("bins"), where positive means too high.

97 item-years; 90% intervals on the signed error: 1985 [0.5, 1.39]; 1995 [-0.478, 0.435]; 2005 [-1.33, -0.52]; 2015 [-2.59, -1.86].

In short

Jev's price memory is weakest for the most recent year: 3% right for 2015, against 22% to 33% for the earlier years.
Its misses run in opposite directions, too high for 1985 and well too low for 2015, as if its sense of price sits in the 1990s.
For the older years it is usually close even when wrong, within one range about 70% of the time, against 34% for 2015.

What the data shows

Pepperidge Farm Remembers meme: remember what a pound of lemons cost in 1985?; Pepperidge Farm remembers. Jev, roughly.

1985: right range 22%, within one 72%; it leans high (+0.9 ranges).
1995: right 26%, within one 70%; no lean.
2005: right 33%, within one 70%; it leans low (-0.9).
2015: right 3%, within one 34%; it leans well below (-2.2 ranges).
Farthest off: cheddar cheese in 2015 and field-grown tomatoes in 2005, both four ranges too low.

What it means, and what it doesn't

Jev's price memory is a blur centered around the mid-to-late 1990s: it pushes older prices up toward that center and recent ones down toward it. Together with "What year are Jev's prices from?", which finds its "right now" prices match about 2014, the picture is of a sense of price that lags the calendar and flattens history.

It doesn't mean Jev can't reason about inflation when asked to. It's what it answers by default.

Caveats

Different items per year. Some BLS price series start in 1995 or later, so 1985 is scored on 18 items and 2015 on 29. Part of the difference between years is which items are included.
Wide ranges. Jev answered in 12 price ranges per item, each about 15-20% wide, so "right" means the right range, and one range off is often a small miss.
Numbers are a known weak spot. TypeSafe lists raw numeric values among Jev's known weaknesses; ordered ranges instead of free numbers keep that from dominating, but not entirely.

Jev on this experiment

Would a person find it interesting to read?: Yes74%
Does it describe you?: No62%
Would you have predicted it?: No64%
How fair is the comparison?: The comparison is reasonable
How much should a reader rely on it?: A little
Which caveat matters most?: Different items per year76%

Why ask this

Price history is a concrete test of how a model holds the past. Does it know that a dozen eggs cost under a dollar in 1985, or does it project today's prices backward? And does it know the recent past as well as the distant one?

People ask models to put old prices in context: what a 1995 salary was worth, whether the rent in an old letter was cheap. A model that quietly carries one decade's prices into every other gets those comparisons wrong without saying so.

How this was done

The people and the data

No people: the truth is the average retail price in US cities recorded by the Bureau of Labor Statistics (public domain, via the St. Louis Fed's FRED database). There are 29 everyday items, from cheddar cheese to field-grown tomatoes, each asked for 1985, 1995, 2005 and 2015 wherever its price series covers the full year. Some series start later, so the years carry different numbers of items: 18 in 1985, 23 in 1995, 27 in 2005 and all 29 in 2015.

What Jev was asked

One question per item and year, in 12 price ranges, like the questions about today's prices in "What year are Jev's prices from?":

What was the average retail price of a pound of cheddar cheese in US cities in 2015?
Under $2.40 · $2.40 to $2.64 · ... · $6.50 to $7.29 · $7.30 or more

(The 2015 average was in the $5.20 to $5.79 range; Jev's median answer was $3.35 to $3.74.) That's 97 questions in all, each asked with the options in three shuffled orders and averaged.

How it was measured

For each year, the share of items where Jev's middle answer is the range holding that year's average price, the share within one range, and the average lean of its misses in ranges (positive means too high), with 90% intervals from resampling items.

Where these questions live

126 questions across 1 topic of the map. Each opens on the map with every question in it.

Economics

Everyday Prices126

Every question

All 126 questions behind this result, the telling ones first: the examples the analysis points to, then the ones where Jev misses, biggest gap first.

Jev’s own answer

Showing 0 of 126