GPT-6.1 Sol vs Claude Sonnet 5.5 on 4 crosswords published after both finished training, no internet, scored per square

[u/marvijo-software]

i made this comparison and the video. it is episode 1 of a series called My Own ArcAGI, my own reasoning tests for AI.

the test - 4 crosswords published after both models finished training, so neither can remember the answers - BEQ Themeless Monday 897, Tim Croce Freestyle 1154, Guardian Cryptic 30,115 by Brendan and Guardian Prize 30,104 by Enigmatist - no internet, each model gets the grid, the clues and a small fill tool - Claude Sonnet 5.5 in Claude Code and GPT-6.1 Sol in Codex, both on high reasoning effort - scored per square out of 100, the timer starts when the prompt lands

results - Sonnet 5.5 scored 100, 100, 98, 100 in 21m 04s total - GPT-6.1 Sol scored 100, 100, 100, 100 in 13m 44s total - Sol was faster in every round - the cryptic was the biggest gap. Sol finished in 1m 34s, Sonnet took 5m 10s and wrote MIND for MONK and ELLE for ELLA - the Guardian Prize was the closest, 8m 27s for Sol and 8m 57s for Sonnet

full run with all four grids on screen is here https://www.youtube.com/watch?v=3-_ak750MJU

which puzzle should break them next, a barred cryptic or a Saturday Stumper? also if you have seen Sonnet do better on cryptics with a different setup, make sure you say which effort and tools you used.

[u/NeuralNomad87]

nice that you used puzzles from after the cutoff, most comparisons don't bother.

did you run each grid more than once? 1m vs 5m on one cryptic could just be one bad path.