> Try bucketing clock time vs. blunders or castling analysis or opening vs. ELO on 1 billion PGNs
No one uses PGN for that. Strawmen are easy to defeat.
Heck, I can crush your example by simply using a proper binary format, fixed length records, and reading flat files directly into memory, all for about 1 hour of coding to transcode your format into something designed for analysis.
Better yet, compare your examples to any of the current best chess DB formats. They all support all your query examples a massively more. And are properly designed, tested and improved over decades of actual use, by professionals and researchers alike.
People have been doing billion+ chess game analyses for well over a decade. Chess.com added 1 billion games in Feb alone two years ago , they just passed 100 billion games total in Jan, they regularly run all sorts of analyses on all the games, and provide datasets and services to researchers for similar work.
It's hard to improve on what you do not understand. If you spent some time understanding the space you may be able to improve on it. Until then a lone wolf, unaware of what is in the space, will simply de-invent a worse wheel.
No one uses PGN for that. Strawmen are easy to defeat.
Heck, I can crush your example by simply using a proper binary format, fixed length records, and reading flat files directly into memory, all for about 1 hour of coding to transcode your format into something designed for analysis.
Better yet, compare your examples to any of the current best chess DB formats. They all support all your query examples a massively more. And are properly designed, tested and improved over decades of actual use, by professionals and researchers alike.
People have been doing billion+ chess game analyses for well over a decade. Chess.com added 1 billion games in Feb alone two years ago , they just passed 100 billion games total in Jan, they regularly run all sorts of analyses on all the games, and provide datasets and services to researchers for similar work.
It's hard to improve on what you do not understand. If you spent some time understanding the space you may be able to improve on it. Until then a lone wolf, unaware of what is in the space, will simply de-invent a worse wheel.