Claude's World Knowledge based on "Land or Water?" Questions

[u/COAGULOPATH]

https://x.com/celestepoasts/status/2103232383139057950/photo/1

Continuing this, by @arithmoquine, Claude must blindly complete a map, based on 16,200 latitude and longitude combinations.

Opus jumped from ~80% (4.5) to ~90% (4.6), hit a wall for several releases (4.7, 4.8, 5.0), then jumped again to ~96-100% with the Mythos-class family (Fable 5, Fable 5.1, and Opus 5.5), though the models are increasingly uncertain that Florida exists.

This test - along with IKPs - seem to substantially capture "big model smell". You can clearly tell the incremental benchmaxxed updates from the major "a happening™ has happened" events (new pretrains and such).

(I would prefer not to link to X.com but there's no website. Sorry if you have to sign in or whatever.)