Atlas › What it knows

Case study 26 of 198

Jev answers country questions with 'the richer one'

Asked which of two countries has more doctors, internet users, unemployment or smokers, does Jev know the numbers, or lean on which country is richer?

result8,137 questions

Asked to compare two countries on a World Bank statistic, Jev is right 92% of the time when the answer is the one wealth would predict (the richer country has more doctors, the poorer one more farming), but 80% when the real answer goes against that pattern. The split is widest for doctors per person (95% vs 45%), the share of the economy in farming (96% vs 52%) and taxes (88% vs 50%).

0%25%50%75%100%

physicians

agriculture share

tax share

electricity access

mobile subscriptions

alcohol

exports share

infant mortality

life expectancy

co2 per person

remittances share

overweight

inflation

precipitation

gdp total

aged 65 share

forest cover

renewable electricity

protected land

tobacco use

unemployment

against itfits the wealth rule

How to read this: Each row is one statistic. The hollow square is how often Jev is right when the answer follows the rich-country pattern, the filled square how often it's right when the answer goes against it. A long gap means Jev leans on wealth instead of knowing the number.

8,137 pairs over 21 indicators; 90% intervals [0.912, 0.923] and [0.788, 0.819]. Pairs against the pattern are closer on average, so the split is partly about closeness: on pairs at least 2x apart it is 93% vs 86%. Jev picks the richer country 63% of the time; the right answer is the richer one 64% of the time.

In short

Comparing two countries on a World Bank statistic, Jev is right 92% of the time when wealth predicts the answer, and 80% when it doesn't.
The shortcut is starkest for doctors per person: 95% right when the richer country has more, 45% when it has fewer.
Pairs that break the pattern also tend to be closer in value; on pairs at least twice apart the gap shrinks to 93% against 86%.

What the data shows

92% right when the answer fits the pattern, 80% when it doesn't.
The rule of thumb shows most on: doctors per person (95% when the richer country has more, 45% when it doesn't, a coin flip), farming's share of the economy (96% vs 52%) and taxes as a share of the economy (88% vs 50%).
Jev picks the richer country about as often as it should: 63% of the time, when the richer country is the answer 64% of the time. The lean shows only on the hard exceptions.
Closeness explains part of it: on pairs at least twice apart, the split narrows to 93% vs 86%.

What it means, and what it doesn't

Jev knows a lot about countries, but some of that knowledge is a shortcut: when it's unsure, "richer country, more doctors" stands in for the real number. For anyone using a model for quick country facts, the exceptions are where to check.

It doesn't mean Jev is guessing everywhere: on about half the indicators it stays above 80% even against the pattern (life expectancy 85%, CO2 per person 84%, remittances 87%), and on a few it does better against it than with it (unemployment 89% vs 56%, tobacco use 85% vs 61%). On doctors, farming and taxes, the exceptions are close to a coin flip.

Caveats

Close calls sit against the pattern. Pairs that break the wealth pattern tend to be closer in value, and close calls are harder anyway. On pairs where one value is at least twice the other, the split shrinks to 93% vs 86%. The rule of thumb is real, but part of the 92% vs 80% gap is closeness.
Who counts as rich. Wealth here is GDP per person, known for 150 countries; pairs involving other countries are left out.
Averages over years. The values are World Bank averages over recent years (2019 to 2023). A country that changed fast can make an honest answer look wrong.
Politics left out. Comparisons touching contested political topics were flagged and excluded.

Jev on this experiment

Would a person find it interesting to read?: Yes79%
Does it describe you?: No71%
Would you have predicted it?: No57%
How fair is the comparison?: The comparison is reasonable
How much should a reader rely on it?: Moderately
Which caveat matters most?: Close calls sit against the pattern99%

Why ask this

Most facts about countries follow money. Richer countries have more doctors per person, longer lives, fewer people working the land. A model can get many country comparisons right just by knowing which country is richer, without knowing the actual numbers.

The test is the exceptions: a poorer country with more doctors per person than a richer one, or a richer one that collects less in taxes. When the world goes against the pattern, does Jev still get it right, or does it fall back on the rule of thumb? Someone asking a model a quick country fact can't tell which one they got.

How this was done

The people and the data

No people: the answer key is the World Bank's World Development Indicators (CC BY 4.0), averaged over 2019 to 2023. 8,137 pairs over 21 indicators, from doctors per person and farming's share of the economy to CO2 per person and rainfall, each pairing two countries on one indicator. Wealth is GDP per person, known for 150 countries, so for every pair it is clear whether the right answer is the one wealth predicts. Some indicators, like rainfall, barely follow wealth at all and act as a check. Comparisons touching contested politics were left out.

What Jev was asked

Two-option questions like:

Which country has a larger share of its population aged sixty-five or older: New Zealand or Jamaica?
Jamaica · New Zealand

Each pair was also asked with the two countries in the other order.

How it was measured

The 8,137 pairs are split by whether the right answer is the country wealth predicts, and Jev's accuracy is compared in the two groups, overall and per indicator, with 90% intervals.

Where these questions live

8,137 questions across 22 topics of the map; the 16 biggest are shown. Each opens on the map with every question in it.

Every question

All 8,137 questions behind this result, the telling ones first: the examples the analysis points to, then the ones where Jev misses, biggest gap first.

Jev’s own answer

Showing 0 of 8,137