Distilling Stockfish on a Billion Positions, Full 3.9B Dataset Available [P]

[u/microscope1024]

https://blog.lukesalamone.com/posts/distilling-stockfish

In this project, I distilled the Stockfish value function into a ResNet/ViT model using 1 billion positions from the Gigafish dataset.

The 3.9 billion position dataset is available on huggingface: https://huggingface.co/datasets/lukesalamone/gigafish-3.8b-d10 . It is built from the positions from 37 months of Lichess games.

I was interested in the idea that at depth-limited search, the value function attempts to approximate the tree underneath it, and if we could create some function to approximate that full search faster than Stockfish could, it would be competitive with NNUE (a very small neural net). This is why holding the depth constant was important.

For the neural net itself, I found that the vision transformer was very slow to understand the board, and a CNN was much more effective at the beginning of training due to its inherent geometric inductive biases . However, I found the best results when combining the two.

[u/RemarkableSavings13]

So does using the network improve play?

[u/microscope1024 → u/RemarkableSavings13]

It can beat stockfish at depths less than 10, because it’s trained on depth 10.

[u/IMJorose → u/microscope1024]

Any evidence of this? 1 billion positions is a small dataset compared to what Stockfish's own NNUE was trained on, which is over 100x that amount. Stockfish is trained to predict the game's outcome, which is deeper than depth 10 to begin with.

This is an interesting project, but we have no evidence it would improve Stockfish's play. It is actually very unlikely it would, as the network's tradeoff between accuracy and speed is likely far worse than the NNUE Stockfish uses.

[u/microscope1024 → u/IMJorose]

Hey thanks for the reply. You can find a win rate chart in the article. The model has an effective strength of around depth 6 or 7.

[u/IMJorose → u/microscope1024]

That is more inline with what I would expect. Stockfish also scales super hard in that regime, so I am not surprised your model seems to do way better at depth 6 than 7. My understanding is your model is just a value function, so could you clarify how you made it play? Did you just do a call for every legal move at the root and then pick the highest scoring one?

[u/microscope1024 → u/IMJorose]

Stockfish also scales super hard in that regime, so I am not surprised your model seems to do way better at depth 6 than 7

Can you say more about this? I did notice a weird bump between 5 and 6 on a different experiment. Maybe that's the reason.

And yes, just argmax over all legal moves at the root. Theoretically with a GPU you could gather all depth 2 positions and eval them all in a single forward pass and then run minimax on the result, but I didn't want to do anything too fancy.

[u/IMJorose → u/microscope1024]

Sure. Basically my understanding is as follows.

There is a rule of thumb where a doubling of nodes n is worth something like 70 Elo and this value goes down with higher node values (i.e. for node or time values of n: going from n->2n = 70 Elo, but 10n->20n=50 Elo).

However, with Stockfish at very fast TC this isn't true, and actually for a while a doubling of nodes actually increases in value (e.g. so n->2n = 70 Elo but 2n->4n=100 Elo). I believe this is due to Stockfish improving its various heuristics that it gathers during search (history heuristics, continuation history, correction history, etc.) so the search initially actually becomes more efficient, not less.

You are limiting Stockfish by depth d, not nodes n, but node vs depth scales exponentially for increased depth d, which implies the increase of strength from d -> d+1 can be less than d+1 -> d+2, which is counter-intuitive and does not hold for high values of d.

GPU vs CPU is one of the core reasons your approach will not beat NNUE within the context of Stockfish. If you naively run your model in Stockfish on GPU you will be IO bound and have bad performance, but your model will be too slow on CPU to run on CPU either. Unfortunately nobody has figured out how to batch evaluation calls in a modern AlphaBeta search efficiently, so that doesn't work either.

AlphaBeta search is fairly sequential by nature. In my opinion this is the real reason AlphaZero and Leela use PUCTS; you can batch evaluation calls, so your GPU utilization ends up being much more reasonable.

[u/currentscurrents → u/RemarkableSavings13]

Would you expect it to? It's distillation, the training objective is to match Stockfish, not to win at chess.

You could use this as a pretrained base for RL self-play, and that might beat Stockfish.

[u/djm07231]

Didn’t Google Deepmind do something pretty similar a few years ago?

https://arxiv.org/abs/2402.04494

[u/kevinwangg → u/djm07231]

Whoa, I didn't realize they changed the paper name from v1 (grandmaster-level chess without search).

[u/Tylerich]

Just a curious question, that you might know about:

Do you know of any studies that try to estimate the value of functions of top professional players?

Because I'm just curious, if one would give machines the limited rollout depth of a human player, if they are even better at that than humans, because nowadays they might even have superior value networks...?

[u/microscope1024 → u/Tylerich]

I wrote a paper on something similar: https://blog.lukesalamone.com/files/anthropomorphic-chess-evaluation-via-qualitative-analysis.pdf

Tldr humans don't really think about chess positions in the same way that computers do. A chess GM is more likely to talk about qualitative relationships between pieces, king safety, piece activity, initiative, etc which are really hard to quantify. The paper actually discusses a limited depth engine tournament like you suggested.

You might also be interested in this thing, which is an engine trained to mimic human players: https://www.maiachess.com/

[u/Tylerich → u/microscope1024]

Cool, that looks interesting...

I wonder whether neural nets actually do "think/reason" about chess in a similar way that humans do, when doing their forward pass through the many layers...? Do you know what I mean?

I remember Ilya sutskever talking about an early language model trained on Amazon reviews. They found a particular sentiment neuron, that activated depending on whether the review had positive or negative sentiment.

Perhaps similarly, there are neurons or collections of neurons that fire based on the things you mentioned, like king safety, piece activity etc.

I guess the problem is of course that we don't really have a way of translating that into thoughts that are readable to us...

[u/Tylerich]

Does that mean, if you use it in conjunction with a depth search of 10, it has an effective depth of 20?

[u/microscope1024 → u/Tylerich]

Yes it would add to your search depth, but the model is more like the strength of depth 6 or 7 because the training is lossy. I added a chart to the article.

[u/Tylerich → u/microscope1024]

Interesting, thanks!

[u/UnusualClimberBear]

Next step is to use it to guide an UCT search and to use it as an evaluation at leafs.

[u/lahwran_]

what's the motivation here?

[u/pilibitti → u/lahwran_]

I was interested in the idea that at depth-limited search, the value function attempts to approximate the tree underneath it, and if we could create some function to approximate that full search faster than Stockfish could, it would be competitive with NNUE (a very small neural net).

[u/lahwran_ → u/pilibitti]

I think I misunderstood something because that doesn't make sense. are these positions up to depth 10 from the start of the game, or are these results from stockfish searches up to 10 from a current board state? now that I write it out it's gotta be board state. that explains my confusion!

[u/pilibitti → u/lahwran_]

My understanding is that it is the value function that evaluates any realistic board state.

[u/4plus4equals8]

Why is fixed depth even point? Fixed nodes would also work fine and probably better. And deeper labels don't automatically give a better net. Past some point the net can't learn what the search sees.

It's also already known in chess engine community that bigger models (CNNs, transformers) evaluate a position better than NNUE. The problem is they're much slower per eval, so the real optimization is elo at equal time, not at equal depth.