West of what? Jev picks the city named first

For two US cities, does Jev judge which is farther west by the city, or by its state, the way people do when they place Reno east of Los Angeles because Nevada lies east of California?

result160 questions

Which of two US cities is farther west? The state doesn't mislead Jev (it's right 72% of the time when the state points the wrong way, 63% when it doesn't), but the wording does: it's right 91% of the time when the western city is named first in the question and 44% when it's named second, no better than a coin flip.

western city named first91%

western city named second44%

state misleads (balanced)73%

ordinary pairs (balanced)63%

How to read this: Four bars. The first two split Jev's accuracy by whether the western city is named first or second in the question; the last two by whether the cities' states point the wrong way. The gap in the first pair is the finding.

160 pairs (80 where the state misleads); 90% intervals, named first [0.866, 0.963], named second [0.346, 0.538]. Swapping the two options' order in the answer list doesn't remove it: the lean follows the order in the question sentence, which stayed fixed. No human answers to these pairs exist; the state-misleads pattern is from Stevens & Coupe 1978.

In short

Jev avoids the classic human error of judging east-west by the state, getting 72% right on pairs where the state misleads.
Instead it leans toward whichever city the question names first: 91% right when that city is the western one, 44% when it isn't.
Its surest mistake: 97% sure Chicago is farther west than Mobile, Alabama, which is the western one.

What the data shows

Car Salesman Slaps Roof Of Car meme: *slaps the city named first*; this one's farther west

The state doesn't fool it. Jev is right 72% of the time when the state points the wrong way and 63% when it doesn't. It isn't making the human mistake.
The word order does. When the western city is named first, Jev is right 91% of the time. When it's named second, 44%, no better than a coin flip. Jev mostly just picks the first city.
Swapping the answer buttons doesn't fix it: the lean follows the order in the question sentence.
Its surest wrong answer: 97% sure Chicago is farther west than Mobile, Alabama, which lies about 0.4 degrees west of it.

What it means, and what it doesn't

Jev avoids the classic human trap and falls into a different one: on close east-west calls it leans toward whichever city you mention first. For anyone asking a model to compare two things, the order of the question can decide the answer when the model isn't sure. This kind of position bias isn't on TypeSafe's list of known weak spots.

It doesn't mean Jev's geography is bad in general. North-south comparisons within the US come out right 89% of the time (see "Jev's mental map puts Europe too far south"). East-west gaps here were small, as little as 0.3 degrees, exactly where a guess has to fill in.

Caveats

The question's word order never changed. Each pair was asked with the two cities in one fixed order in the question sentence ("Chicago, Illinois or Mobile, Alabama"). The answer buttons were also swapped, which barely moved Jev. Asking each pair both ways round in the sentence itself is the obvious next test.
No human answers to these pairs. People's classic mistake here, judging by the state instead of the city (Reno feels east of Los Angeles because Nevada is east of California; Stevens and Coupe, 1978), comes from the literature. No one answered these exact pairs.
"The state misleads" is a proxy. A pair counts as misleading when the city in the more western state, by the average longitude of that state's cities, is actually the eastern one. That's a rough stand-in for where a state sits in someone's head.
A data error in the city list. Cities were chosen as US places over 150,000 people in the GeoNames database, but at least one entry (Meads, Kentucky, listed with 288,649 people) is a small community, so the list includes a few places nobody would call well-known.

Jev on this experiment

Would a person find it interesting to read?: Yes69%
Does it describe you?: No63%
Would you have predicted it?: No74%
How fair is the comparison?: The comparison is shaky
How much should a reader rely on it?: Not at all
Which caveat matters most?: The question's word order never changed96%

Why ask this

People store places in a hierarchy (city inside state inside country) and reason from the top down. So they place Reno east of Los Angeles, because Nevada is east of California, when Reno is actually farther west. The psychologists Stevens and Coupe described this in 1978.

A model that reasons the same way will be confidently wrong exactly where the shortcut fails, and people use models for quick geography all the time: which airport is closer, which way a road runs, what time zone a city is in. This experiment asked whether Jev takes the state shortcut, and found something else steering it.

How this was done

The people and the data

The truth comes from coordinates in GeoNames, an open geographic database. The project picked US cities of over 150,000 people in different states, between 0.3 and 5 degrees of longitude apart, and built 160 pairs: every pair where the state misleads (the city in the more western state, judged by the average longitude of each state's cities, is really the eastern one), 80 in all, and 80 ordinary pairs. There are no human answers to these pairs; the human side is the published pattern.

What Jev was asked

Which city is farther west: Chicago, Illinois or Mobile, Alabama?
Answers: Chicago, Illinois · Mobile, Alabama

The answer buttons were asked in both orders and averaged. The order of the two cities in the question sentence stayed as it was built.

How it was measured

The share of pairs Jev gets right, split two ways: by whether the state misleads (balanced so each group has as many "western city first" pairs as "western city second"), and by whether the western city is named first or second in the question.

Where these questions live

160 questions across 1 topic of the map. Each opens on the map with every question in it.

Continents & Regions

Where Cities Lie160

Every question

All 160 questions behind this result, the telling ones first: the examples the analysis points to, then the ones where Jev misses, biggest gap first.

Jev’s own answer

Showing 0 of 160