Your network blocks the Lichess assets!

lichess.org
Donate

Science of Chess: The AI Revolution Hasn't Made Chess Better (yet!)

Thanks for your regular posts. Often the greatest difficulty is reducing your claim to a series of single metrics. What does it mean to make chess better, as in terms of individual metrics, separate from human play, even top level human play. I have mentioned in many of your posts, from the research perspective, there is very little to gain from studying top players, the main research benefit of chess is the process of acquiring expertise, especially the beginning levels.

Within that you need two separate metrics, one for how AGI has changed the understanding of the game itself, more in a matter of game theory, and the search for Truth. I would argue that there have been significant advances such as now understanding that chess is likely drawn in most major variations, and that Black can get an equal game in almost any opening. And other harder to quantify new understandings such as time value relationship of material and other factors beyond calculation. It is likely there are countless individual advancements in the 'objective search for truth', that only computers, logicians and mathematicians can solve. As the search for Truth in chess is better determined by the computer and mathematicians / coders than chess players.

From the human level, AI has caused many differences in human play, most notably by demonstrating the limits of human mind. So from a human level, competitive chess at the highest level is less about the search for Truth, but understanding human mental limitations, and how to incorporate that into competitive play. Like 'what does it mean for a position to be equal but easier for a human to play?' So now top competitive play leans against an objective search for truth, but more recognizing the limits of the human mind, and top play being more about recognizing and capitalizing on the limits of your and your opponents knowledge. A more advanced form of game theory now involved in top competition, where understanding the 'truth of the position' is secondary to understanding the limits of the human mind. Most notable, might be the computer beating top GMs with odds, leading to new understanding of game theory as applies to chess.

And most notable, with advances in AI, not just with chess, but across the board, is the diminishment of the value of expertise. Top players have much less to offer newer or improving players, and training techniques and methods are likely to become more dominated by the computer, and the value of top players becomes less. This will likely also diminish the value and social standing of top chess players, who become more questionable the investment of time into chess for the diminishing returns of chess mastery.

To me, the main areas of research are not directly related to chess, but to the acquisition of mastery, and the main question is there a universal method to mastery, and about transfer of expertise. It is possible to divide the question for AGI and humans, as the answer for humans and AGI might be different. As with Alpha Zero, a universal algorithm that applies to all fields, and can transfer, might be evidence towards a universal path to expertise and transfer of expertise. But likely that the human mind works different than AGI, so even if it can be mathematical / computer / game theory demonstrated from AGI that there is a universal path to mastery, and that mastery can transfer, certain limitations of the human mind mean that for humans there is not only no universal path to expertise, but also that expertise does not transfer from field to field among humans.

So the main benefit for researchers from AGI, is evidence of the limitations of the human mind, possibly leading to new discoveries and theories for consciousness and cognitive functioning.

Thanks for your regular posts. Often the greatest difficulty is reducing your claim to a series of single metrics. What does it mean to make chess better, as in terms of individual metrics, separate from human play, even top level human play. I have mentioned in many of your posts, from the research perspective, there is very little to gain from studying top players, the main research benefit of chess is the process of acquiring expertise, especially the beginning levels. Within that you need two separate metrics, one for how AGI has changed the understanding of the game itself, more in a matter of game theory, and the search for Truth. I would argue that there have been significant advances such as now understanding that chess is likely drawn in most major variations, and that Black can get an equal game in almost any opening. And other harder to quantify new understandings such as time value relationship of material and other factors beyond calculation. It is likely there are countless individual advancements in the 'objective search for truth', that only computers, logicians and mathematicians can solve. As the search for Truth in chess is better determined by the computer and mathematicians / coders than chess players. From the human level, AI has caused many differences in human play, most notably by demonstrating the limits of human mind. So from a human level, competitive chess at the highest level is less about the search for Truth, but understanding human mental limitations, and how to incorporate that into competitive play. Like 'what does it mean for a position to be equal but easier for a human to play?' So now top competitive play leans against an objective search for truth, but more recognizing the limits of the human mind, and top play being more about recognizing and capitalizing on the limits of your and your opponents knowledge. A more advanced form of game theory now involved in top competition, where understanding the 'truth of the position' is secondary to understanding the limits of the human mind. Most notable, might be the computer beating top GMs with odds, leading to new understanding of game theory as applies to chess. And most notable, with advances in AI, not just with chess, but across the board, is the diminishment of the value of expertise. Top players have much less to offer newer or improving players, and training techniques and methods are likely to become more dominated by the computer, and the value of top players becomes less. This will likely also diminish the value and social standing of top chess players, who become more questionable the investment of time into chess for the diminishing returns of chess mastery. To me, the main areas of research are not directly related to chess, but to the acquisition of mastery, and the main question is there a universal method to mastery, and about transfer of expertise. It is possible to divide the question for AGI and humans, as the answer for humans and AGI might be different. As with Alpha Zero, a universal algorithm that applies to all fields, and can transfer, might be evidence towards a universal path to expertise and transfer of expertise. But likely that the human mind works different than AGI, so even if it can be mathematical / computer / game theory demonstrated from AGI that there is a universal path to mastery, and that mastery can transfer, certain limitations of the human mind mean that for humans there is not only no universal path to expertise, but also that expertise does not transfer from field to field among humans. So the main benefit for researchers from AGI, is evidence of the limitations of the human mind, possibly leading to new discoveries and theories for consciousness and cognitive functioning.

@DIAChessClubStudies said ^

Thanks as always for your comments! I've made a few responses below.

Thanks for your regular posts. Often the greatest difficulty is reducing your claim to a series of single metrics. What does it mean to make chess better, as in terms of individual metrics, separate from human play, even top level human play. I have mentioned in many of your posts, from the research perspective, there is very little to gain from studying top players, the main research benefit of chess is the process of acquiring expertise, especially the beginning levels.

I've written about this before (https://lichess.org/@/NDpatzer/blog/science-of-chess-should-we-study-gms-to-understand-chess-and-the-mindbrain/vjhM2y3K) and agree with you to some extent. My only caveat is to say that this is just a different kind of research question that is specifically about performance at the top-levels. I'd argue that this is still a case where chess confers a research benefit insofar as we're using it as an instance where technological advancements may or may not have driven human advances.

Within that you need two separate metrics, one for how AGI has changed the understanding of the game itself, more in a matter of game theory, and the search for Truth. I would argue that there have been significant advances such as now understanding that chess is likely drawn in most major variations, and that Black can get an equal game in almost any opening. And other harder to quantify new understandings such as time value relationship of material and other factors beyond calculation. It is likely there are countless individual advancements in the 'objective search for truth', that only computers, logicians and mathematicians can solve. As the search for Truth in chess is better determined by the computer and mathematicians / coders than chess players.

OK, so now it's my turn to mention that I don't think it's all that useful to cast work like this in the context of AGI. Why general intelligence and not domain-specific cognition? At any rate, I think there's a wide range of metrics one can choose to consider if you want. Are the ones used here perhaps too reductive? I'd say no (though I'm open to seeing a similar analysis of other richer metrics) but I also wouldn't be too fussed if you disagree.

From the human level, AI has caused many differences in human play, most notably by demonstrating the limits of human mind. So from a human level, competitive chess at the highest level is less about the search for Truth, but understanding human mental limitations, and how to incorporate that into competitive play. Like 'what does it mean for a position to be equal but easier for a human to play?' So now top competitive play leans against an objective search for truth, but more recognizing the limits of the human mind, and top play being more about recognizing and capitalizing on the limits of your and your opponents knowledge. A more advanced form of game theory now involved in top competition, where understanding the 'truth of the position' is secondary to understanding the limits of the human mind. Most notable, might be the computer beating top GMs with odds, leading to new understanding of game theory as applies to chess.

This I very much agree with and I think it's fascinating. I quite like the potential for looking at players of different strength and determining something like the "negative space" of ideas that are hard to have.

And most notable, with advances in AI, not just with chess, but across the board, is the diminishment of the value of expertise. Top players have much less to offer newer or improving players, and training techniques and methods are likely to become more dominated by the computer, and the value of top players becomes less. This will likely also diminish the value and social standing of top chess players, who become more questionable the investment of time into chess for the diminishing returns of chess mastery.

This is a neat perspective, but I think in some ways top players are becoming almost interpreters between tne engines and weaker players. I think that means mastery still has value, but different value than it did.

To me, the main areas of research are not directly related to chess, but to the acquisition of mastery, and the main question is there a universal method to mastery, and about transfer of expertise. It is possible to divide the question for AGI and humans, as the answer for humans and AGI might be different. As with Alpha Zero, a universal algorithm that applies to all fields, and can transfer, might be evidence towards a universal path to expertise and transfer of expertise. But likely that the human mind works different than AGI, so even if it can be mathematical / computer / game theory demonstrated from AGI that there is a universal path to mastery, and that mastery can transfer, certain limitations of the human mind mean that for humans there is not only no universal path to expertise, but also that expertise does not transfer from field to field among humans.

I think chess research by and large supports the idea that there isn't a meaningful universal method to mastery at all. This is why I'm so skeptical about the utility of AGI as a framework for thinking about the mind.

So the main benefit for researchers from AGI, is evidence of the limitations of the human mind, possibly leading to new discoveries and theories for consciousness and cognitive functioning.

Thanks again for sharing your thoughts!

@DIAChessClubStudies said [^](/forum/redirect/post/WWfPI0vr) Thanks as always for your comments! I've made a few responses below. > Thanks for your regular posts. Often the greatest difficulty is reducing your claim to a series of single metrics. What does it mean to make chess better, as in terms of individual metrics, separate from human play, even top level human play. I have mentioned in many of your posts, from the research perspective, there is very little to gain from studying top players, the main research benefit of chess is the process of acquiring expertise, especially the beginning levels. I've written about this before (https://lichess.org/@/NDpatzer/blog/science-of-chess-should-we-study-gms-to-understand-chess-and-the-mindbrain/vjhM2y3K) and agree with you to some extent. My only caveat is to say that this is just a different kind of research question that is specifically about performance at the top-levels. I'd argue that this is still a case where chess confers a research benefit insofar as we're using it as an instance where technological advancements may or may not have driven human advances. > Within that you need two separate metrics, one for how AGI has changed the understanding of the game itself, more in a matter of game theory, and the search for Truth. I would argue that there have been significant advances such as now understanding that chess is likely drawn in most major variations, and that Black can get an equal game in almost any opening. And other harder to quantify new understandings such as time value relationship of material and other factors beyond calculation. It is likely there are countless individual advancements in the 'objective search for truth', that only computers, logicians and mathematicians can solve. As the search for Truth in chess is better determined by the computer and mathematicians / coders than chess players. OK, so now it's my turn to mention that I don't think it's all that useful to cast work like this in the context of AGI. Why general intelligence and not domain-specific cognition? At any rate, I think there's a wide range of metrics one can choose to consider if you want. Are the ones used here perhaps too reductive? I'd say no (though I'm open to seeing a similar analysis of other richer metrics) but I also wouldn't be too fussed if you disagree. > > From the human level, AI has caused many differences in human play, most notably by demonstrating the limits of human mind. So from a human level, competitive chess at the highest level is less about the search for Truth, but understanding human mental limitations, and how to incorporate that into competitive play. Like 'what does it mean for a position to be equal but easier for a human to play?' So now top competitive play leans against an objective search for truth, but more recognizing the limits of the human mind, and top play being more about recognizing and capitalizing on the limits of your and your opponents knowledge. A more advanced form of game theory now involved in top competition, where understanding the 'truth of the position' is secondary to understanding the limits of the human mind. Most notable, might be the computer beating top GMs with odds, leading to new understanding of game theory as applies to chess. This I very much agree with and I think it's fascinating. I quite like the potential for looking at players of different strength and determining something like the "negative space" of ideas that are hard to have. > > And most notable, with advances in AI, not just with chess, but across the board, is the diminishment of the value of expertise. Top players have much less to offer newer or improving players, and training techniques and methods are likely to become more dominated by the computer, and the value of top players becomes less. This will likely also diminish the value and social standing of top chess players, who become more questionable the investment of time into chess for the diminishing returns of chess mastery. This is a neat perspective, but I think in some ways top players are becoming almost interpreters between tne engines and weaker players. I think that means mastery still has value, but different value than it did. > > To me, the main areas of research are not directly related to chess, but to the acquisition of mastery, and the main question is there a universal method to mastery, and about transfer of expertise. It is possible to divide the question for AGI and humans, as the answer for humans and AGI might be different. As with Alpha Zero, a universal algorithm that applies to all fields, and can transfer, might be evidence towards a universal path to expertise and transfer of expertise. But likely that the human mind works different than AGI, so even if it can be mathematical / computer / game theory demonstrated from AGI that there is a universal path to mastery, and that mastery can transfer, certain limitations of the human mind mean that for humans there is not only no universal path to expertise, but also that expertise does not transfer from field to field among humans. I think chess research by and large supports the idea that there isn't a meaningful universal method to mastery at all. This is why I'm so skeptical about the utility of AGI as a framework for thinking about the mind. > > So the main benefit for researchers from AGI, is evidence of the limitations of the human mind, possibly leading to new discoveries and theories for consciousness and cognitive functioning. Thanks again for sharing your thoughts!

Be interesting to study to mediating effects of human mastery to understand computer mastery. There would be multiple ways to test this (such as having players of different skill levels compete each with similar computer aid). But some estimate say that a 2700 can better mediate the computer wisdom than lower gradations of expertise. And whether other areas of expertise, such as mathematics, logic, coding,..., might help more mediate the computer than chess expertise.

I would aim more for a game theory / logic approach as more valuable than top players mediation. If there is a 'mathematical / logical' answer to chess, the opinions of top players should have no bearing, and the need to consult them limits the theory.

The main benefit of research on chess is that it is a closed well defined game, but has near infinite patterns far beyond the calculation skills of the human mind, and 'memorizing / internalizing' so many patterns that human mastery takes years without any short cuts.

About the universal path to mastery - if there is a universal algorithm (like used in alpha zero) that can be demonstrated, but yet the mind can not replicate the results of the machine, what are the implications of how the mind works?

Be interesting to study to mediating effects of human mastery to understand computer mastery. There would be multiple ways to test this (such as having players of different skill levels compete each with similar computer aid). But some estimate say that a 2700 can better mediate the computer wisdom than lower gradations of expertise. And whether other areas of expertise, such as mathematics, logic, coding,..., might help more mediate the computer than chess expertise. I would aim more for a game theory / logic approach as more valuable than top players mediation. If there is a 'mathematical / logical' answer to chess, the opinions of top players should have no bearing, and the need to consult them limits the theory. The main benefit of research on chess is that it is a closed well defined game, but has near infinite patterns far beyond the calculation skills of the human mind, and 'memorizing / internalizing' so many patterns that human mastery takes years without any short cuts. About the universal path to mastery - if there is a universal algorithm (like used in alpha zero) that can be demonstrated, but yet the mind can not replicate the results of the machine, what are the implications of how the mind works?

Never forget Geller-Panno 1955! https://www.quora.com/Have-there-ever-been-identical-chess-games-played-in-professional-chess-tournaments-Whats-the-closest-case

In my mind, the next logical question is: how can we visualize how chess opening trees changed over the years? Maybe I should make this a Lichess backlog issue, although making a useful tool for teaching purposes seems like a large research project, similar to one or more research projects cited in your blog post. I pray that some reader can sort out the implementation details, so I can copy someone else's homework assignment instead of doing it myself.

Never forget Geller-Panno 1955! https://www.quora.com/Have-there-ever-been-identical-chess-games-played-in-professional-chess-tournaments-Whats-the-closest-case In my mind, the next logical question is: how can we visualize how chess opening trees changed over the years? Maybe I should make this a Lichess backlog issue, although making a useful tool for teaching purposes seems like a large research project, similar to one or more research projects cited in your blog post. I pray that some reader can sort out the implementation details, so I can copy someone else's homework assignment instead of doing it myself.

@Toadofsky said ^

In my mind, the next logical question is: how can we visualize how chess opening trees changed over the years?

Lichess does it already for individual lines. I think if you do a statistical analysis on how openings changed, you will see those moments of change after PC engines and AlphaZero. @NDpatzer's analysis specifically excluded the first 10 moves, though.

@Toadofsky said [^](/forum/redirect/post/TaqYJJaO) > In my mind, the next logical question is: how can we visualize how chess opening trees changed over the years? Lichess does it already for individual lines. I think if you do a statistical analysis on how openings changed, you will see those moments of change after PC engines and AlphaZero. @NDpatzer's analysis specifically excluded the first 10 moves, though.

@NDpatzer , I believe the ability to learn from experience is a personal quality that can't be captured at a statistical level. Having the most powerful computer engines at my disposal doesn't guarantee I use them to discover my failure points and certainly doesn't mean I will make any decent effort of fixing those. Learning has been, is and it will always be hard.

The study was smart, it identified individual players and tried to discover change points in their evolution, but I don't think that accuracy is a meaningful metric. It basically asks "did Magnus blunder less when better computers were available?".

And there are so many changes in the data that separating meaningful effects is impossible. The player number exploded with online play, the playing style changes based on fashion, the popular platforms and who YouTubes best at the moment. And all of these trends are enabled and directed by advances in technology.

Bottom line: maybe Morphy would have been a god of chess if presented with computers, but maybe he would have become addicted by social media instead. Maybe Magnus fundamentally changed his perspective on chess, but his opponents did too. And also, no amount of training would make me play like him anyway.

My point is that chess evolution is deeply personal and statistical methods can't scry inside our brains yet.

@NDpatzer , I believe the ability to learn from experience is a personal quality that can't be captured at a statistical level. Having the most powerful computer engines at my disposal doesn't guarantee I use them to discover my failure points and certainly doesn't mean I will make any decent effort of fixing those. Learning has been, is and it will always be hard. The study was smart, it identified individual players and tried to discover change points in their evolution, but I don't think that accuracy is a meaningful metric. It basically asks "did Magnus blunder less when better computers were available?". And there are so many changes in the data that separating meaningful effects is impossible. The player number exploded with online play, the playing style changes based on fashion, the popular platforms and who YouTubes best at the moment. And all of these trends are enabled and directed by advances in technology. Bottom line: maybe Morphy would have been a god of chess if presented with computers, but maybe he would have become addicted by social media instead. Maybe Magnus fundamentally changed his perspective on chess, but his opponents did too. And also, no amount of training would make me play like him anyway. My point is that chess evolution is deeply personal and statistical methods can't scry inside our brains yet.

@TotalNoob69 said ^

In my mind, the next logical question is: how can we visualize how chess opening trees changed over the years?

Lichess does it already for individual lines.

How do I see how opening trees changed over the years?

@TotalNoob69 said [^](/forum/redirect/post/FdNcSONY) > > > In my mind, the next logical question is: how can we visualize how chess opening trees changed over the years? > > Lichess does it already for individual lines. How do I see how opening trees changed over the years?

@TotalNoob69 said ^

@NDpatzer , I believe the ability to learn from experience is a personal quality that can't be captured at a statistical level. Having the most powerful computer engines at my disposal doesn't guarantee I use them to discover my failure points and certainly doesn't mean I will make any decent effort of fixing those. Learning has been, is and it will always be hard.

The study was smart, it identified individual players and tried to discover change points in their evolution, but I don't think that accuracy is a meaningful metric. It basically asks "did Magnus blunder less when better computers were available?".

And there are so many changes in the data that separating meaningful effects is impossible. The player number exploded with online play, the playing style changes based on fashion, the popular platforms and who YouTubes best at the moment. And all of these trends are enabled and directed by advances in technology.

Bottom line: maybe Morphy would have been a god of chess if presented with computers, but maybe he would have become addicted by social media instead. Maybe Magnus fundamentally changed his perspective on chess, but his opponents did too. And also, no amount of training would make me play like him anyway.

My point is that chess evolution is deeply personal and statistical methods can't scry inside our brains yet.

I definitely get this perspective, but I also think that changing the scale from the personal to the historical means that we're talking about fundamentally different things, for many of the reasons you named. I'd also say that accuracy is a meaningful metric, but can't hope to capture everything about how gameplay changes over time. That's fine, and almost always the case when you have to choose some way to operationalize a complex phenomenon.

That said, there's a neat thing that happened with the arrival of AlphaZero in that our understanding of what optimal moves were had to change! That evolution doesn't just affect one person - if we find out that there's a fatal flaw in some variation, for example, or that certain kinds of aggressive moves are definitely worth making, that knowledge becomes part of the chess community. To me, that's what this kind of analysis is about: How does the "field" of chess (like a scientific field) incorporate new knowledge/

I agree that there are lots of other sources of variability to consider and also lots of richness to the game that's being elided here, but I'd still say that the statistical look is worth doing and interesting to think about.

@TotalNoob69 said [^](/forum/redirect/post/Xt2W9R05) > @NDpatzer , I believe the ability to learn from experience is a personal quality that can't be captured at a statistical level. Having the most powerful computer engines at my disposal doesn't guarantee I use them to discover my failure points and certainly doesn't mean I will make any decent effort of fixing those. Learning has been, is and it will always be hard. > > The study was smart, it identified individual players and tried to discover change points in their evolution, but I don't think that accuracy is a meaningful metric. It basically asks "did Magnus blunder less when better computers were available?". > > And there are so many changes in the data that separating meaningful effects is impossible. The player number exploded with online play, the playing style changes based on fashion, the popular platforms and who YouTubes best at the moment. And all of these trends are enabled and directed by advances in technology. > > Bottom line: maybe Morphy would have been a god of chess if presented with computers, but maybe he would have become addicted by social media instead. Maybe Magnus fundamentally changed his perspective on chess, but his opponents did too. And also, no amount of training would make me play like him anyway. > > My point is that chess evolution is deeply personal and statistical methods can't scry inside our brains yet. I definitely get this perspective, but I also think that changing the scale from the personal to the historical means that we're talking about fundamentally different things, for many of the reasons you named. I'd also say that accuracy is a meaningful metric, but can't hope to capture everything about how gameplay changes over time. That's fine, and almost always the case when you have to choose some way to operationalize a complex phenomenon. That said, there's a neat thing that happened with the arrival of AlphaZero in that our understanding of what optimal moves were had to change! That evolution doesn't just affect one person - if we find out that there's a fatal flaw in some variation, for example, or that certain kinds of aggressive moves are definitely worth making, that knowledge becomes part of the chess community. To me, that's what this kind of analysis is about: How does the "field" of chess (like a scientific field) incorporate new knowledge/ I agree that there are lots of other sources of variability to consider and also lots of richness to the game that's being elided here, but I'd still say that the statistical look is worth doing and interesting to think about.

@NDpatzer said ^

To me, that's what this kind of analysis is about: How does the "field" of chess (like a scientific field) incorporate new knowledge/

I totally agree. Personally, I like analysing chess positions a lot more than actually playing any :D

@NDpatzer said [^](/forum/redirect/post/8zfViwg7) > To me, that's what this kind of analysis is about: How does the "field" of chess (like a scientific field) incorporate new knowledge/ I totally agree. Personally, I like analysing chess positions a lot more than actually playing any :D