Hypothesis, Movement, Procedure
Psychological studies of chess are not only cool, but if we are lucky they can suggest practical ways to improve. Readers will be familiar with this wonderful study that helped me out a great deal (discussed in practical terms here).Here I discuss two studies that may also have practical implications. One is a nice paper that someone recently sent me. The other has been on my 'Blog This' pile for a year or so now.
1. Candidate move evaluation as hypothesis testing: amateur confirmation bias
The first paper, Chess Masters Hypothesis Testing is extremely interesting. In looking at chess players' thought process, they fruitfully frame candidate moves as hypotheses. The hypothesis, for each candidate move is: "This is the best move in this position." What is the best way to test this hypothesis for a given candidate? Obviously you must consider how your opponent will respond (especially sharp moves such as checks, captures, and other threats). Unlike science, where the data are measurable quantities like voltage and mass, the only data we have to go on are variations we calculate and patterns/procedures stored in our memory.
If you'll forgive a brief digression, an early and common view of hypothesis testing in science was that we should go out and seek confirmation. E.g., if you think F=ma, then look for instances in which that is true. Observe it enough times, and you have good scientific support for your hypothesis.
But that isn't often the best way to do things. It suffers from a 'confirmation bias', our tendency to look for data that confirms our theories, much like a UFO conspiracy theorist who sees every unexplained light in the sky as being controlled by little green man. It is like a person that believes God created all species that seizes on any gaps in our scientific knowledge of biological speciation as evidence for intelligent design.
In contrast, a good scientist will do her damnedest to kill her own pet theories, to falsify her hypotheses. They try to find the most unexpected, surprising prediction and do the experiment. After a while, those theories that remain, that survive these falsification attempts, are taken much more seriously.
And that is what we should do in chess. When considering a candidate move, we have to consider the worst possible outcome, we have to put ourselves in our opponent's shoes and try to kill that move. If we just look at moves we'd like our opponents to make, we are not being objective, and will get demolished. Sure, it might lead to mate in three if he plays like a jackass on crack, but if he plays like a human being, you will get destroyed. So be objective.
Getting back to the study, the researchers studied thought processes of amateur versus master-level chess players, and they found a significant confirmation bias in the amateurs. Specifically, when analyzing candidate moves the amateurs tended to spend their time thinking about subvariations that were good for them, but overlooking variations that refuted that line. The master-level players were much more objective, able to quickly see the bad aspects of a move as well as the good aspects.
That is really cool.
We have known for some time that in chess, the least bad move is the best move. So while it is good to have positive plans and strategies, once time is running and you have settled into the nitty-gritty of evaluating a particular candidate move, hold nothing back, attack it with all your intellect. The candidate that is least bad is the best.
Amateurs, beware the tendency to see UFOs where there are just flying turds.
Note also I think this confirmation bias is ubiquitous in the chess improvement literature. There are not enough objective studies of chess improvement, so you end up with chess improvement clans. What we need are chess improvement data and studies, not anecdotes and emotion anchored in a cult of personality.
2. Learning 10,000 Pictures
The second study, Learning 10,0000 Pictures, is just amazing. Subjects were shown more than ten thousand pictures in one session, viewing each picture for five seconds, and saw each image only once. Two days later, subjects were tested for their recall of the images (some they had been shown previously, some not). They were able to accurately recall over 6500 of these pictures! The experimenters didn't even try to train the subjects on more images in this heroic experiment, but simply concluded "the capacity of recognition memory for pictures is almost limitless."
Holy shit. So why do we have so much trouble remembering a few thousand chess positions? If we remembered positions the way we remember more natural visual images, we'd all be much better at chess: we could scroll through a bunch of positions or problems and remember them without a lot of trouble (especially if we did the Circles on them!).
Why are we so much worse with chess positions? I think there are two reasons. I didn't mention before that subjects did much worse when the task was to recognize words, and much much worse if they were shown nonsensical strings of letters. So it seems important that the images be of things you would tend to actually see in real life rather than abstract names or symbol strings. Chess seems to be more like language than real life in this regard--abstract symbols that you wouldn't exactly encounter in nature were it not for human cultural scaffolding.
If only there were some way to translate chess information into naturalistic images in our brains, so we could better remember the former.
The second difference between the study and our chess learning is in the task itself. Subjects only had to recognize the images, not do anything with them that could be construed as right or wrong. In chess, pattern recognition is not enough. I have had many instances of remembering a pattern in a tactical puzzle book, but having no idea of the solution even though I had worked through it before. Pattern recognition and solution recognition are different species. Indeed, I don't care if I recognize the original position as long as I remember the bloody solution!
It seems there are two components to chess mastery--yes, pattern recognition which humans are very good at (though better with realistic images than symbol strings). That is fairly passive, and happens without any effort (like you recognize your mom's face). But there is also a more active component--the ability to do something. I can recognize the dials and gizmos in an airplane cockpit, but I have no idea what to do with them. Becoming a good pilot, so you are flipping the dials like an old pro, takes a great deal of experience; no matter how much book knowledge you have it just takes a great deal of time to develop the procedural expertise. In other words, it is a motor skill only partly guided by pattern recognition.
Chess seems to be the same way--sure it helps to recognize patterns, but it is even more helpful to just move your hands to the right piece and move that piece to the right square with minimal thinking. That is, you need to build up procedural memory, and that just takes a lot of time, and you can't force it no matter how smart you are.
What can we do with this? Well, one thing to do is follow Temposchlucker. Like Fox Mulder, he is getting closer to the truth than he has ever been in his recent post that touches on this selfsame topic. Tempo and I have been orbiting this topic for over two years, and I think this study sheds some light on the topic (that post was the first one where I brought up and defined the distinction, from psychology, between procedural and declarative memory).
But what would falsify this hypothesis about motor skills in chess? To be objective, it is crucial to think in those terms: otherwise I'm no better than a new-agey Creationist UFO hunter. Can someone with severely impaired motor skills, such as a quadriplegic, excel at chess? If a theory implies that Stephen Hawking couldn't get good at chess, the theory is in need of revision. If so, does that falsify this motor skill based hypothesis? What types of procedural learning is there besides specifically motor learning? Not all actions involve flexing muscles. I can learn to do mathematical problems in my head by practicing them, until I can do them quickly without thinking. That is a type of procedural learning that may be more relevant for chess mastery.
posted by Blue Devil Knight at 4/21/2008 06:01:00 PM
20 Comments:
Certainly alot of things that could be discussed here but with my limited time this am, I will ask you this. Do you think there is a difference between physical manipulation (moving the pieces) and visualizing their movement ?(which is more akin to what happens in a game). Perhaps visualization may be even stronger.
Another big question for me was the danger of calculating incorrect lines and how this incorrect calculation was often associated with the original pattern for me. Perhaps a line of thought would be how to best associate the correct solution to the pattern without interference of incorrect solution.
perhaps there should be a chess program on the screen that shows 2 boards: one with the original position on the left that is static and one to the right that is dynamic that automatically shows the movement of pieces.
The simplest way to falsify a hypothesis is to put it to the test in practice. Since I'm not a scientist any other way is tricky. I use the word "motorskill" but maybe I actually mean some other brain function. When matters aren't clearly defined, as they are within the circles of amateurs, any theoretical treatise will contain flaws.
Just experimenting and letting prof. Elo be the judge prevents those flaws.
It's a good point you bring up again. There are two methods of improvement which have different goals:
Self-explanation by means of narratives to educate the conscious part of the brain and motorskill training to educate the procedural part of the brain.
that objectivity part of it is what always gets me, it's what i need to work on most, killing my best move to prove it works or not.
as for the motor skills needed, i don't know if actually touching the piece and moving it is as crucial as knowing WHAT piece to touch and move, and i don't think that requires actual motor skills, just thought. look at michael aigner, aka fpawn. he is in a wheelchair, severely reduced motor skills, and is a bad ass chessplayer.
excellent topic...
Tak: I agree that visualizing the movement of pieces is probably more important than physical piece movement (this is more like solving the math problems in one's head). I also think that as you improve, visualizing piece movement takes less effort.
But of course it is possible, and perhaps likely, that motor control is important, that swapping around pieces physically helps one to remember the movement of the pieces, to improve visualization. That would be pretty easy to study empirically I think.
Also, good point about misremembering! Having a false memory of the solution so I actually remember the wrong thing to do! That happens a lot. Once I started using FOVEA (linked in first para of post) that stopped happening. I think that has mostly to do with what you emphasize during training--sitting there staring at the problem (then you will tend to remember false lines you originally thought of), or actively working with the solution, making sure you understand it thoroughly.
Excellent post!
But I hate to tell you the aliens are gray and they are not smaller than us.
But I digress...
I thought it was a motor skill thing too. But I think the motor skill portion of it is more of a calculation thing. Robert has a link to an article on Silman's site that discusses this in more detail. I also spoke about it in my last post. Although I do believe the motor skill portion is important, it's not quite as viatl as calculation.
I believe you are correct when you talk about the language of chess. Knowing the basic rules and some of the general principles (bihsops like open positions etc...) is like knowing spanish at freshman level in H.S. Donde esta la biblioteca? Me llamo Pedro. You know really basic stuff.
To achieve mastery however you have to know chess language the way you'd have to be familiar with Spanish to write Don Quixote. This analogy works with any foreign language. Chess is univerally foreign to everyone, as it is a new language to anyone who doesn't know the game.
Tempo:
A. Procedure versus motor
Procedural knowledge is the standard term to contrast with explicit (declarative) knowledge, though it may have started out being called motor skill or something.
And don't give up too soon on your motor skill theory (it is key to try to falsify a putative falsification! :)). Even if it isn't the only thing, perhaps motor practice is extremely helpful for building procedural knowledge. MDLM has us touch the squares, perhaps that helps. I bet a quadripeligic would probably learn chess more slowly than normal subjects. Also, even kids, you teach them by having them throw the pieces around. Try to teach kids without letting them literally interact with the pieces. Would they be better or worse at the game? This is something we could examine experimentally.
And also, it isn't like the quadrapeligic has zero motor skills. To communicate the moves they have to move their eyes or something. Are such motor skills sufficient? Do we have to be able to move our hands and arms, does that matter? Saccades (quick eye movements) are very important, revealing a great deal about one's chess expertise, and interestingly we typically don't consciously control them! This is very interesting to me. (Discussed here and I think we chatted about it recently at your blog).
I wish I had the time to study chess scientifically. One ELO change is an anecdote, a possible stasitical aberration. That's why I like your list of rating changes in the Knights. Even better would be a hundred ELOs of two groups of people in controlled conditions. Luckily studies like the one I cited are trickling in.
But ultimately since the science of chess improvement is in its infancy, we are stuck experimenting on an N of one. It is nice to have people to test our ideas on.
1. Pattern recognition plus....?
You are right there are a few issues here. The self-explanation part you bring up is very helpful (partly for reasons I mentioned above in my response to Takchess), and for me helped me gain procedural knowledge of tactics more quickly. This isn't an accident. As we've been talking about a while now (since the Baars stuff), the two seem connected.
Also, I'm sure it is more complicated than I discuss in here. For example, I said there are two components to chess expertise (pattern rec/procedural knowledge), but my main point was that there is more than just pattern recognition.
Many present day books written by armchair psychologist GMs are still stuck on pattern recognition as the sole key to chess expertise. Clearly that is important and necessary, but not sufficient. Never put all your trust the expert to tell you how he does something--he hardly ever knows himself as most of his knowledge is procedural!
If it were just pattern recognition, we could learn 6000 chess positions a day (give or take).
But there is more to it than the 'pattern plus procedure' mantra. Where does confirmation bias fit in? That's addressing thought process.
But thought process also becomes procedural knowledge (as I discuss here.
But that is almost trivially true to say, since everything worth doing in chess should become procedural knowledge. I do believe that thought process is an important component here (which is why I kept pushing Phaedrus to include thought process in his flowchart).
So the question becomes, what skills are worthy of putting in the immense effort required to place that skill in procedural memory? Good thought process? Definitely--you don't want confirmation bias and can train it away so you automatically are a better thinker in games. Tactics? Of course. Visualization? (Note, even how the pieces move--this was MDLMs genius--those drills aren't about pattern recognizion they are about some of the most elementary procedures).
Once people agree on the procedural knowledge people need, and I think there is probably broad agreement about it even if it isn't called that, the question becomes the best order to focus on procedure-learning (e.g., tactics first), and more importantly the best way to learn those procedures you have already decided are important. These are things you have focused on (that's why I said 'Follow Tempo').
My bias? Focus on the simple until there is absolutely no question that you have that shit down cold. Obviously tactics is the most important thing initially
(well, perhaps tactics after learning how the pieces move, which sounds trivial but we all had to put that in procedural memory--at first I had to sit there thinking..OK how does this piece move? That is, it was in declarative memory and I had to think it through consciously and inefficiently)
CL: Great example with fpawn! I also have a hard time being objective in selecting moves. I prefer to think "Well, if this works it will be freaking cool, and if not I'll learn from it" rather than bunker down and really think. This is especially a problem in blitz for me.
Wang: Thanks for the tips about the posts. I'll go check them out. Chess is indeed a universal foreign language! :)
“Amateurs, beware the tendency to see UFOs where there are just flying turds.” Priceless :)
On the topic of falsifying a candidate move: I couldn’t agree more. I think it’s crucial for improvement, and I actually started doing this a while back after going over games with a strong player at my club. When analyzing games we’d go into some variation where he’d make a move (with either color) and say: prove to me that I can’t make this move! That phrase stuck, and I try to prove to myself that I can't make a particular move on almost every candidate move I look at nowadays.
On the “Learning 10,000 Pictures” study and your question “So why do we have so much trouble remembering a few thousand chess positions?”:
I am probably saying the same thing as you are saying, but in a slightly different way. I think you can make the argument that ONE chess image is actually comprised of multiple “sub-images”. Any given chess position is usually so complex that the brain might not be able to process it as ONE chunk of information, but has to break it down into several bits. I don’t know what level of complexity the pictures used in the study had, or what constituted a successful “recall” of a picture in that study. But I would venture to say that the recall of a picture might have been called successful even if some elements in the participant’s memory of the picture were off. In a chess position though, a recall would be unsuccessful if you get as little as a pawn’s position wrong – it could tremendously alter the entire meaning of that position. There are too many dynamic relationships between the various pieces and squares for a chess position to be as easily “recallable” as a picture.
Great post - it seems to me that your writings get better and better since you announced your retirement ;-)
I really enjoyed the comparison between recognizing pictures and chess positions. It never really occurred to me while studying chess tactics that in normal life, not even horses hop around like the knight. To make things worse, the board geometry is weird, too, because it takes the king the same number of moves to go from a1 to h1 as it takes him to move along the diagonal to h8. That simple fact still messes my endgame calculations up big time...
Concerning the amateur's tendency to prefer variations favoring himself while ignoring possible counters: I think that there are many factors playing into this. Firstly, if I get an idea, it means that I see a path to, let's say, win a piece. These ideas usually "pop up" in my head, caused by pattern recognition. My pattern recognition system, however, is far from complete (otherwise I would be unbeatable) and probably misses a few of the important variations that are so dangerous to me. At the lower level, this behavior even gets reinforced by weak defenders on the other side of the board and simply the before mentioned fun factor of making crazy attacking moves.
Chessaholic: Very interesting stuff. Good questions about the 10,000 images. I think they didn't put similar, but not identical pictures in the testing set. That is a very interesting idea to try, and I'm quite sure subjects would do much worse with such a variant. E.g., at one extreme a picture of a crowd, but then on test day show the same picture but with one person removed from the crowd :) Chess is like that, with a medium-sized crowd.
Scirius: Good point about the strange geometry of the board. It really is counterintuitive because geometric distance isn't the same as chess distance (number of tempi to reach a square).
I think your second point is right. Weaker players also reward my sloppy candidate move selection.
Also, it just feels more natural to see what piece I will gain, to focus on what I want. Only later do I learn patterns for countering standard moves (e.g., I learned the hard way that when the opponent castles queenside it is often a mistake to capture on a7 with a Bishop, because if he then goes b7 your bishop is trapped and likely to get gobbled by the King--at first I just saw the pawn I would gain, the weakening of his kingside--now I know to look for the bishop trap). This seems to be an understandable and probably inevitable progression of pattern learning.
And of course, in blitz games it is hard to do any but the most superficial process of elimination.
In my previous comment, I said:
if he then goes b7 your Bishop is trapped
I mean b6, not b7, trapping the Bishop.
Following up on my comment about the progression of pattern learning (first, learn the positive tactic, how to gain the material or mate or or control the weak square--whatever the goal is; then, build up knowledge/patterns of how they can be standardly refuted).
It would be cool to have tactics software that explored a dozen variations on one theme. The board looks roughly the same, but subtle differences change the playability of the tactic. So, don't just teach you the positive goal, but also show you typical refutations. Would be cool to do a Circles like thing with this. First time through, learn the main tactic. Second time through, do them again, but they also bring up a new board that shows you some typical refutations. Something like that could be pretty cool.
This exemplifies the type of post that makes your blog a must read. BDK, I beg you. Please don't go!
Bill
Hey Bill, thanks :)
This is the PBS-style I'm most comfortable with, that's for sure.
If I find cool chess studies I'll be sure to mention them to Temposchlucker :)
I know better than to say "never", but I just want to wrap things up and take an indefinite break.
Your digression potshot at the evolution debate is not appreciated. Go ahead and consider that your argument means that localized evolution is not evidence for macro evolution, which would make all the non-creationists howl in indignation. Science is for testing the observable...the creation of the world, however it occurred, isn't observable. So let's consider all the possibilities without prejudice (surprise! like a scientist). As strawman arguments to further your chess points I'm sure the readers get it.
As for objectivity in chess...even Fischer wasn't totally objective. It was well known that he preferred bishops over knights.
The link http://www.lincolnchessfoundation.org/dvlpyngchssmstrs.html has some excellent points. Aside from calculation and visualization, don't rule out mentoring, practice, and commitment just to make it to the 98th percentile. Study chess systematically, and expect that it will take anywhere from 8-12 years to become a Master. Fischer had played for years until he, "...just got good."
--Otis
Otis: Your chess views seem quite reasonable (that link you put there is busted, but I would like to track it down).
As for the digression, it isn't that (macro) evolutionary biology isn't good science, but that it is a historical science. We don't directly observe star formation (it takes millions of years) but with the help of observed snapshots of the process at different stages we are able to construct good models. The same is true of evolutionary hypotheses (e.g., land mammal-whale transitional fossils, H. erectus fossils, the distribution and abundance of such fossils, the suite of molecular evidence).
Even if the Creationists are right (and obviously, I think they are not right, either theologically or in their take on science), they are going about things all wrong if they ever hope to be taken seriously. Among (many) other serious problems with their approach, they suffer from a horrible confirmation bias, rarely looking for ways to falsify their theories. That was the main point of the digression.
I don't want to enter a lengthy debate about the specifics here, as I know from experience it would become an infinite time sink. I'll agree we disagree, though, and look forward to more chess stuff.
One ELO change is an anecdote, a possible stasitical aberration
LOL. My rating has proven to be quite resilient against any method of improvement, so if it skyrockets and doesn't come down within 3 years it cannot be mistaken for a statistic anomaly:)
A hypothesis can never be proven to be true, it can only be falsified. This means that you need a bias to formulate a hypothesis. Without the bias of thinking that an adult can improve dramatically with the right method, I wouldn't be able to make such efforts as I did the past 3 years. You can't play the SM without a bias.
But a confirmation bias is quite different. That is a real crime to me since evidence is actually counterfeited.
Having a preference for an outcome of an experiment is a sin to me. If one day is proven that an adult cannot improve dramatically in chess then my world doesn't collaps. I then just know what I want to know.
BTW, I just wrote a post about pattern recognition being overrated as cause for difference in chess performance.
Tempo: LOL. You rock dude.
The problem with statistics is they apply to populations, while what we need is information that applies to individuals. The best we can do is find what seems reasonable, whether it be anecdotal, based on a study whatever, and then experiment on ourselves.
Agreed on the disagreement.
Apparently the link insists on the capitalized letters. This one works in my browser(s).
http://www.lincolnchessfoundation.org/DvlpYngChssMstrs.html
I haven't done a full review of your Chess Planner yet, but I noticed that the biblography is pretty light on pre-2000 resources. I recommend picking up some CJS Purdy (a fmr World Correspondence Champion) and you may find that he had the structured thinking market pegged.
--Otis
Otis: thanks. I've seen that, it is a very interesting article. I had lost it though, and it had become something I referenced but without citation, so it is good to have the ref again.
I've talked about Purdy who has influenced Chessplanner greatly, especially indirectly through Rosario and his book 'A First Book of Morphy'.
For instance, I discuss him here. And you can see his influence quite obviously here in the condensed version of Chessplanner (the first step of the condensed version is basically Purdy).
To cut the manuscript down I may have cut out some older references, especially if I felt someone else said something more clearly than someone else. Again, something we've discussed here before (e.g., here).
See also the last section of the PDF where I discuss someone who might say "There is nothing new here."
Also, you are clearly Blogotype Number 4.
Post a Comment
<< Home