Wednesday, October 24, 2007

More on wins versus losses, reward versus punishment

Sven posted some very interesting comments on my previous post in which I argue that it is important to go over your wins (both to find mistakes to correct and to find what you did well). I think his comments are useful and respond to them here. His comments are in bold, my reaction in normal text: note I edited the order of some of his comments to make the flow more smooth. I like this topic because it is very practical. I think no research has been done on it in chess.

Everyone has ones own method and there is nothing that works for everyone.

Exactly. So why do so many instructors tell everyone to focus only on losses? My gripe is that the psychology of learning is clear on this: there are methods, often involving positive reinforcement rather than punishment, that work better for the majority of people. Why should chess be different? Is it a unique kind of learning problem for which techniques in other areas will not transfer? I don't believe that.

However, it is because of individual differences that in my previous post I said everyone needs to find the mixture of reward and punishment that works best for them. I just urge you to not trust your introspection about what works best for you. Do some experiments on yourself. You may be wrong about how you learn best, partly because of the dogmas perpetuated by the Russian taskmasters that have turned into self-limiting improvement habits. Chess players tend to like to beat themselves up, but is that the most efficient way to improve? See my post on blunderstanding for a slight twist on this. Indeed, I think this tendency amongst chess players may be a simple selection bias: that's how instructors work, so people who don't like those methods tend to not want to go far in chess (in fact, perhaps girls would play more if we used learning paradigms that were more in line with modern psychological theory than the methods used in the Gulag).

If you look at the mistakes in your losses then you can realize sooner which mistakes are important and which are not. A difference that makes no difference (even if it could have) is no difference.

An interesting point. Let's consider why it is so useful to analyze our own games. It is because we will learn the ins-and-outs of positions that we are likely to see in the future: similar pawn structures, piece placement, and therefore tactics and strategies. Plus the emotional investment in the game makes it more likely you will remember what you have learned than when you are simply looking at compositions or puzzles from books.

So, given that, and the fact that I make significant tactical and positional errors in my wins (and not 1900-level errors that I don't need to worry about, but patzer errors: just look at the previous win I posted), it can only help me to study wins. I will learn elementary strategic and tactical patterns that are much more likely to recur in the future. And in the future, my opponent is likely to see the tactical opportunity I gave him, to exploit my positional blunders (especially if I am improving and am more likely to see better players in future games!).

Perhaps the better you get, the less you need to study your wins.

If you win, you might be tempted not to take them seriously as obviously the outcome of the game was in your favour even with the mistake being made.

I don't have this temptation. My hunch is that if you go over wins and see major errors (like Squirrelchess did) you will take them very seriously. Try it and see. If you just get bored and hate going over wins, then you shouldn't do it as this is all for fun. But you should realize you are missing out on some great learning opportunities.

Also, using your reasoning, you'd never be motivated to study annotated master games, as the outcome of the game is completely irrelevant. Again, this is fine if you don't like doing it, but many espouse the benefits of such study (see Heisman's most recent post for instance).

Finally, imagine not reviewing a game in which you were getting destroyed until the last move, where you were able to win via a crazy one-move mate swindle. It would be crazy not to review that game. So the question isn't whether to analyze wins, but which wins to analyze. I am bad enough that every game has lots to learn from. This brings us to the same point as the previous bit, maybe the better you are, the fewer wins you'll need to analyze.

Another point: The goal to get rid of mistakes ("subtract negatives" like Heisman says) is by itself a "negative task", so this is at the core of its being. If one uses a "positive method" one will avoid "regions of negative feelings" when using it and thereby avoid the core of the task.

I am not saying to avoid looking at negatives. It is incredibly important to understand mistakes!! I am saying that is only half the story (or perhaps 3/5 of the story), that for learning from one's games it is also important to reinforce and understand the good moves, especially the happy accidents (this is the essence of reinforcement learning: random exploration of behavioral space, with selection via reinforcement of those behaviors that lead to good consequences).

Incidentally, the word 'punishment' is not really a good description of what we are doing when finding mistakes: as with blunderstanding, we don't have to punish ourselves for mistakes, but can use a different more positive psychological orientation to the negative chess moves. The issues of reward/punishment and mistake/good move focus are orthogonal and I will have to think more about these distinctions.

In my next post I'll talk about why I think it is just easier (but not good) to focus exclusively on mistakes.

10 Comments:

Anonymous Anonymous said...

I grew up studying mathematics, so my mindset is to concentrate on the mistakes I made in the 'test', not go back over the 'questions' I got right. I think chess has a lot of similarities with solving equations etc, but the other thing is that the solutions are so complex that imagination and creativity are sometimes more important than brute calculation. So perhaps the positive reinforcement works better for the creative aspects?

10/24/2007 12:05:00 PM  
Blogger Temposchlucker said...

I can't deny that solving 100k+ problems has caused a certain emotional flatness when looking at a chess position. I mean, if you sac 30 queens a day, no. 31 will be not very exiting any more.

Emotion plays a role in learning, that is for sure. That positive emotions work better than negative ones according to science I take for granted. But if a triggered emotion will be positive or negative is highly dependent on how ones ego is structured.

To give an example, I can become very enthousiast when I discover a major flaw in my game. That is because I see behind it a major possibility to improve. But I'm sure not everybody sees it this way.

Margriet gets down when she looks at her losses. So we study only her won games, and I stress the good things while relativating and ridiculizing the bad ones. No problemo. That's why she still plays chess. (and most of your guys wifes not:)

in fact, perhaps girls would play more if we used learning paradigms that were more in line with modern psychological theory than the methods used in the Gulag

I'm sure you hit the nail on the head with this one.

10/24/2007 12:13:00 PM  
Anonymous Anonymous said...

BDK,

First of all, thanks for your extensive answer in your posting. I'll put my comments to your points under the quotation of it (remarks in [] within those quotes are from me). Again, it's long (no complaints from my side if you just scroll it down, I will try to be shorter in the future), but this time there is not so much disagreement in it and there is some clarification of some points. I really think we only differ on small things. It mainly is an explanatory comment, but of course I'm not at all opposed to discussion should it come up anyway.

"I don't have this temptation [to not take mistakes in won games seriously]. My hunch is that if you go over wins and see major errors (like Squirrelchess did) you will take them very seriously. Try it and see."

First, it's good you don't have this temptation, because I do.

If I didnt realize mistakes immediately after I played them, then I take them very seriously even in my wins (because it is a clear indication of a serious flaw in my thought process). But if I saw them afterwards I made them and it didnt have consequences I just think: "This shouldn't have happened" and move on. If I lose because of such a thing I think the same, but it's far more intensive (when there is already -say- one hour of the rest of the game when I go down the losing road (maybe my team as well) and can think "This _really_ shouldn't have happened") and because of that I'm motivated after the game in a much stronger fashion to get rid of this. If its in a won game or even if it's in a lost game and my opponent makes no use of it, it's just like: "ok, mistakes happen. Now better watch out." Less motivation- for me at least.
And if I win I dont want to (mainly) think about the mistakes I made, like: Ok, I won, but had my opponent been a 1900 I surely would have lost. In my wins I try to learn more from the things that worked for me and in the losses more from the things that didnt.
Another reason for this is: I dont fully understand what is a good move and what is a bad move and if I win, "something that clearly led to this" was a good move whereas in a loss "something that clearly led to this" is a mistake. So its an extension of the "bad move/good move database". So, for "unknown errors" and "unknown good moves" this approach might help.

Of course if a mistake in a win catches my eye that is as important as something working out well in a loss. But it doesn't leave such a lasting impression and thus doesn't create such a great motivation to really learn from it. But that may be different from person to person and it certainly is best to learn from all mistakes and all good moves, however they occured.

"If you just get bored and hate going over wins, then you shouldn't do it as this is all for fun. But you should realize you are missing out on some great learning opportunities."

I agree that it is better to look at wins than not to do it. If not for ones mistakes than for ones successes. I like to see myself win!

"Also, using your reasoning, you'd never be motivated to study annotated master games, as the outcome of the game is completely irrelevant. Again, this is fine if you don't like doing it, but many espouse the benefits of such study (see Heisman's most recent post for instance)."

To be honest, I dont see how this would be a result of my reasoning. The mistake in a won game was irrelevant in so far as it didnt have a loss as a consequence. The outcome of the master game is only in that regard irrelevant as i try to learn from the moves but not from the result ("Lasker-Steinitz 1-0, now I see" :) ). It is relevant concerning the question what is succesful.
I think, master games are important indeed, especially to see which kind of chess works and which doesnt. They also show how many different ways of playing there are and that there can be spectacular, tactical/sacrificial wins against high class competition as well (if one looks at those "old games", like end of 19th, beginning of 20th century, there are really fantastic combinations against people that would today still be at maybe 2500 - this was a revelation, at least for me, who was thinking before that: "After reaching 1800 people always play positional chess, unless the opponent blunders").

"I am not saying to avoid looking at negatives. It is incredibly important to understand mistakes!! I am saying that is only half the story (or perhaps 3/5 of the story), that for learning from one's games it is also important to reinforce and understand the good moves, especially the happy accidents"

I agree with that. Maybe I misunderstood your first posting. I thought you were stating something along the lines of (the following is no quote): "Its better to use a positive reinforcement and to focus on ones good moves than focusing on ones bad moves. If you look at mistakes, rather do it within your wins". Now I think you talk more about the problem of _exclusively_ dealing with mistakes in losses. I still think there is more to be learnt from mistakes in ones losses than in ones wins, but I concede that its better to look at ones mistakes in won games too.

"(this is the essence of reinforcement learning: random exploration of behavioral space, with selection via reinforcement of those behaviors that lead to good consequences)"

Wow, I'll have to think about that (after I understood it :) ).

"The issues of reward/punishment and mistake/good move focus are orthogonal and I will have to think more about these distinctions. "

So do I. Im not really convinced of my argumentation concerning "negative core" vs "positive method". Sounds too esoteric and like an excuse for a lack of stringent argumentation. I meant something like: Recognizing a mistake is always a negative experience at first. There is no way around this and when the "surroundings" of this experience are negative (as in a loss caused by this mistake) the negative experience itself is stronger. Because this negative experience is the source of the motivation to advance/improve, there is a greater motivation when the feeling is more negative (and so there is greater motivation from recognizing mistakes that led to losses than from those within wins). That's why I think the problem of getting rid of ones mistakes calls rather for an approach of 'punishment and avoidance' than for one of 'positive reinforcement' (on the other hand this approach seems to lead to a program one could name "masochistic chess improvement". ;) ). There can be positive reinforcement but it has to be created rather artificially as in "Whenever I get rid of my mistakes I get better in the long run. The best method to get rid of my mistakes is to recognize them clearly which is always connected with a negative feeling. So it feels bad, but really is good."

But Im no psychologist or behavioural scientist and so I might use these concepts in an inapropriate way.

And after all there is more in chess than ones own mistakes and the quest to avoid them, otherwise it would be a really depressing endeavour.

Im looking forward to your next posting. It's good stuff for thought you are delivering on these pages regularly.

kind regards,
svensp

10/24/2007 01:04:00 PM  
Blogger Blue Devil Knight said...

Dean: A very interesting analogy. I have a natural tendency also to look at chess positions as logic puzzles with right and wrong answers, which is similar to your view of chess as analagous with math problems.

There are a couple of issues here. One, what methods are most efficient for getting to the point where you can solve problems on your own? Here it is pretty clear that kids learn math better when, in addition to getting "punished" with lower grades, they get positive reinforcement for what they've done well (oh, well that answer isn't right, but look you were looking in the right direction by trying to break it into a division problem, so lets find where you went off track after that so you can get it right in the future).

A test situation isn't really a learning situation, though. It tests where your previous training has gotten you. Sort of like a game of chess. But instead of 1-100 there is only 1 or 0 or .5 grade :) But perhaps like in a math test, for purposes of learning we need to give partial credit: just using 1-0-.5 is a crude evaluation of performance. Someone can play great and then miss a mate in one, or play horribly and get the right answser (a swindle at the end). Those different results require different strategies to overcome them in the future.

Like in math, in chess we often find the right answer for the wrong reasons. These are useful learning opportunities. Just going over losses will miss many of them. (And unlike math, in chess there are often multiple right answers).

And when doing real mathematics (not on a test), much of the time is spent exploring ideas that turn out to be wrong, lots of paper in the trash bin. Look at Temposchlucker. But once the good idea is hit, it really sticks. When a patzer like me accidentally hits on the right answer in chess, it's good to understand it so I can do it again in the future.

Anyway, interesting analogy with math, one I'll need to think about more. It may have some very fruitful consequences for chess improvement, well beyond the scope of my original posts.

Tempo: The more I see into the chess subculture, the more I see why girls hate it.

10/24/2007 01:29:00 PM  
Blogger Blue Devil Knight said...

Dean: I think there are two key questions.

1. What is the most efficient way to improve at getting the right answers during test-time (e.g., tactical, circles, endgame study, postmortem, coach, etc).

That is the most important question for chess improvers.

2. What is the best way to analyze those tests once they are done so that you will do better on future tests?

This is really a subset of number 1. It is possible to improve without doing postmortem, but it is one of the few techniques everyone agrees is good.

In math, when you get 100% on a test, it is usually because you just kicked ass and know the subject really well, at least better than other people in the class. You can't do horribly on the whole test but end up getting a swindle at the end and getting an A. This disanalogy with math tests make studying wins very important for me.

There are few players scoring 100's in chess, even on their wins. That's what I was getting at by the crudeness of the 1-0 score. Perhaps we should score our game by dividing the number of moves in which we played the best move (or within 0.5 or 1.0 or .24 pawns of the best, depending on how good you are), divided by the total number of moves. Using that criterion, there would be few to no 100s in chess, even in wins, and so room for improvement.

Obviously, that won't change who won or lost, but in the postmortem that isn't important. It's how well you did from play to play that gives the learning opportunities. A series of mini-tests, some of which you scored well on, some of which you get zero credit.

My hunch is that people are in a habit of not studying their wins (partly justified by their coach or something they read), and now there is inertia and they don't have time or feel "lazy" (though they are not actually lazy: just ask their wives if their husbands are lazy when it comes to chess).

10/24/2007 01:45:00 PM  
Blogger Blue Devil Knight said...

Sven: I obviously don't mind long comments. There are a lot of issues here, and this stuff isn't discussed all that much to my knowledge in books or in blogs.

I also think we agree to a large degree. I never meant to imply that positive reinforcement should replace looking at mistakes. That would be a huge mistake! :)

Perhaps I was wrong about the master games thing, but let me push it a bit more. You said "The outcome of the master game is only in that regard irrelevant as i try to learn from the moves but not from the result ("Lasker-Steinitz 1-0, now I see" :) )"

But why would you ever do that? Would you not look at a beautiful Kramnik game in which a computer mated him in one on the last move? My previous comment to Dean, on a potentially better scoring system for postmortem analysis, may apply.

The equation is:
[1 -(number of moves less than best)/(total number of moves)].

It will be a score between 0 (when all of your moves are less than best) and 1 (all your moves are best). This is what you should use to determine whether to analyze the game, not the crude 1-0 score! If your score is 0.1 but you scored the full point, you are doing a great disservice to yourself.

So in effect, I think you should be punishing yourself more. :)

10/24/2007 01:57:00 PM  
Blogger Grandpatzer said...

The gist of what I'm seeing is that sometimes learning has to be fun.

The solution as I see it is: learn how to have fun analyzing your losses and enjoy finding weaknesses in all your games--wins and losses. Like a body builder might exercise all muscle groups to the point of fatigue, regardless of whether they enjoy bench presses more than squats.

Easier said than done? Maybe. But after an upsetting loss I actually find the process therapeutic.

10/24/2007 02:41:00 PM  
Anonymous Anonymous said...

BDK,

"Perhaps I was wrong about the master games thing, but let me push it a bit more. You said "The outcome of the master game is only in that regard irrelevant as i try to learn from the moves but not from the result ("Lasker-Steinitz 1-0, now I see" :) )"

But why would you ever do that? Would you not look at a beautiful Kramnik game in which a computer mated him in one on the last move?"

Of course I would. I literally meant the result as in "1-0". The last move is as important as any other move (or probably more).

"My previous comment to Dean, on a potentially better scoring system for postmortem analysis, may apply.

The equation is:
[1 -(number of moves less than best)/(total number of moves)].

It will be a score between 0 (when all of your moves are less than best) and 1 (all your moves are best). This is what you should use to determine whether to analyze the game, not the crude 1-0 score!"

Hm, I'd rather analyze a game with one huge mistake than 5 smaller ones, but if you put in a weight for the gravity of each mistake, I would agree- in theory. Objectively you are right with this. Still, it's a fact, I get a greater motivation when an error causes me to lose. Of course the error in a winning game is no less important than the one in a losing game. But here's a thought experiment: Say there is an error that would never (guaranteed!) result in a loss- would you care to correct it? So any important errors will come up in my losing games anyway as causes of a loss and if they don't then they obviously have no influence on the outcome of the game. And those mistakes that appear more often as the cause of a loss are also more important, for they determine most of the time the change of outcome. Equally those ways to play are most important which are mostly the cause of a win and other good moves are not as important.
But its better to analyze both, wins and losses, you are right. And there can be mistakes in wins which can teach one more than mistakes in losses, so ignoring mistakes in wins closes off another valuable source of learning.

"So in effect, I think you should be punishing yourself more. :)"

Now that's easy: I'll simply make more mistakes! Thats the way to improve, I guess. ;)

To the first part of your second posting: What do you think would a method of positive reinforcement look like for removing ones' errors? I understand what it would look like when strengthening good moves/concepts, but removing errors? Maybe the thought process is the point to start. A good thought process can be cherished by a teacher and it should lead to less errors.

kind regards,
svensp

10/24/2007 02:47:00 PM  
Blogger Blue Devil Knight said...

Nice thought experiment sven. Yes, an error that never resulted in a loss would be one I wouldn't care about. I wish my errors were like that.

10/24/2007 02:56:00 PM  
Anonymous Anonymous said...

I can't imagine not going over wins.

I mean; I won the game! An exciting event for me. That's easy; I get to find mistakes that didn't cost me the game (this time!) and so emotionally it's nice for me.

Losses? Now that's depressing. I still make myself go over them, because I know how important it is, but much harder than my wins.

Warped

10/24/2007 08:49:00 PM  

Post a Comment

<< Home