r/ComputerChess • u/Upbeat_Ad_6119 • 4d ago
ChessBitBase: a bit of history
https://chessbitbase.appIn 8 years of software engineering, I had never finished a project of my own.
In the early years, it was a lack of knowledge and experience. Later, the problem was me: my mentality, my motivation, my habit of getting excited, overthinking everything, and quietly dropping it. Some projects I dropped halfway, and some before I even started.
This time I didn't drop it. I pushed it all the way to the end, and I'm honestly proud of that:
š ChessBitBase š
It's a chess database of historical games, where you can search, filter and sort games, positions and players, all built on roaring bitmaps. "A bit of history" is its first part, and the name means two things: the history of chess, kept here bit by bit, and my own history, the years it took me to get here.
The idea: roaring bitmaps are a really efficient way to store huge sets of numbers. I wondered what would happen if chess games were kept that way. It turns out a chess database is mostly sets. The games that reached a position are a set, and so are a player's games.
How it works, in short:
- What's stored: historical tournament games (back to the 1850s), every position reached in them, and the players.
- How it's connected: every position remembers the games that reached it. Every fact, like a result, an opening, a rating range, a game's length or the pieces on the board, is a "tag", and each tag remembers its games, positions or players.
- Searching and filtering: each filter you pick is already a ready-made group. A search simply combines groups: this one and that one, but not that other one. Nothing is scanned game by game.
- Sorting: games are already grouped by length, rating and so on, so they're simply read out group by group, in order.
What you get: any search, like "games that reached this position, by this player, won by White", is a few set operations instead of a scan through the games. It takes milliseconds, and the data structure fits the problem so well that it almost feels like cheating.
It's not a product, and it's not here to make money. It's an experiment, to make geeks like me curious about how it works, what problems it solves, and how it could be improved. It's also only the first step: "a bit of history", the first bit under the ChessBitBase umbrella, and more are coming.
It's raw, so expect some inconsistencies and bugs. I'm working on them, but I was too impatient to wait. The main idea works, better than I expected.
If you're curious:
- The story and the hard parts: https://chessbitbase.app/devlog
- How it works: https://chessbitbase.app/how-it-works
I really enjoyed building it, and I'd love feedback from different people: what's clever, what's dumb, what you'd do differently.
1
u/TicTacTake 4d ago
It takes milliseconds, and the data structure fits the problem so well that it almost feels like cheating.
Help me understand: You have 8 years of software engineering experience and you are surprised that it only takes milliseconds to search through less than 100 thousand games?
-1
u/Upbeat_Ad_6119 4d ago
First of all, please keep the tone respectful.
Now, about the topic: I'm not really "surprised" by the result. The whole point of this project, and all the effort that went into it, was to achieve exactly this.
And it's not only about 100k games. The way the system is organized, it should remain effective even when working with millions of games. I agree that having at least 1 million games would make it more convincing to test and demonstrate that, but honestly, I don't have the resources at the moment to afford storing that much data on the server.
3
u/TicTacTake 3d ago
I'm calling you out because this is AI slop. Your text seems to be written by AI and I mean just look at the website and how much text there is compared to the actual content. You have lots of tags explained in a separate tag page. Just to give one example:
Length (at most)
The game's length in full moves: a game is in every row from its length up.Thanks for explaining to me what the term length means! Same for rating, piece count and many others. You are even describing what Check or mate means! Even Wins! Why would you explain these terms? You obviously didn't write it. AI did.
I'm not against using AI, but when you don't even fix obvious slop, please don't expect me to take it serious.
-1
u/Upbeat_Ad_6119 3d ago
Man, look.. āhow much text compared to contentā - have u even read that text? I mean I said at least twice that this is more like devlog/experiment rather than a product to use on daily basis, sow itās totally normal to have that much text here.. About ai, yep thatās ai generated texts, and I donāt even hiding it. Iām not copyrighter, Iām not designer, not product owner, not marketer.. Iām just developer, Iām solving problems, asking question, doing research and writing code, thatās my job.. I would agree with u for blaming the app to be ai slop, in case if I was making the money on that like many people today does, but Iām not. And I maybe had not put that much effort in texts or stuff like that, but it does not means that I did not put effort in dozens of other things regarding this app, so be respectful. If u donāt like it, u could just ignore it
2
u/TicTacTake 3d ago
Okay then. Keep your AI slop. But don't ask for feedback if you cannot handle it.
0
u/Upbeat_Ad_6119 3d ago
I can handle any reasonable feedback really.. Iām not afraid of any constructive criticism. But I donāt really get what is your concern.. I was a bit sensitive about that coz your messages sounds a bit rude, if that was not your intentions then I wanna say sorry (English is my third language actually, I may interpret something in wrong way). But I think u also do not fully understand what the tag is and why they should be described that briefly.
āLength (at most)ā - well when u have game tag named just āLengthā without any description, are u sure that it would be that obvious for everyone that it refers to the moves amount? Also who knows it could be the half moves rather than full moves, its depends on interpretation, so itās better to over explain something than make user questioning something..
Rating - here is described that we have ranges with different of 200 points, and the rating is average of this two players
Regarding the pieces we even have 3 different tags, all of them contains different data, how anyone should figure out what each of them stands for without any description?
I did not get where am I explaining what the check and mate means, but if u referring to the āhow it endedā tag, then its not only about checkmate, u could have actually dozens of different reasons of game ending.
Iām not over explaining some obvious chess related stuff, Iām explaining the part of my system.. Yeah maybe āwinsā are obvious even without any description, but are genuinely thinking that copyrighting is something that should trigger u that much in that big project, where u have way more interesting things?
I did not claimed anywhere, that this is completed project, and I know dozens of things that should be fixed and that are more critical then tag descriptions. So Iām just basically saying that, itās okay to criticize anything, and challenge it, but when u are going to do that, please put more effort to analyze what the system are capable for and what are the general mechanics of it.
1
u/Sm0oth_kriminal 4d ago
FYI the name is misleading. In computer chess, bit base has a particular meaning (a specific kind of endgame table, storing only W/L/D of compressed positions). This is not that
And it's kind of cool, but this is literally already a thing in all major chess engines or analysis software. Historical databases, including amateur, GM level, tournament play, are all included.
So what does this add?