Monday, February 22, 2010

ICC should post opening stats as a function of rating

I was excited to learn we can track down statistics about the opening at ICC, but then bummed to learn that they only save statistics for matches between the titled players.

It would be cool, helpful, and generally awesome if ICC let us navigate opening statistics for different rating blocks, or for people within X rating points of your present rating.

Such a feature would be a great check for authors of opening books. Often they erroneously pretend that the frequency with which a club player will encounter a move is reflected in the statistics of their database of GM games. Pity the poor club player who studies the book mindlessly, only to find out that in a tiny percentage of his games does he get the chance to actually enter the lines.

For instance, if you study the Marshall defense as black, and are rated below 1500, have fun the one time you actually get to play it in your next thousand games of chess.

Obviously a rating-indexed opening database would be incredibly helpful to the club player. It would let us get a better sense for the variety of play we will encounter in practice, rather than the variety of play we would encounter if we were IMs or GMs. So obvious, why isn't it done? Is it simply the server space that would be needed?

Yes, of course you should play your best moves always and understand principles, and if someone goes off book there is a chance they are violating some principle. That's nice to say and makes you sound smart, but in practice things are typically not so simple and it is hard to refute a line. And if it is easy to refute a line for an IM, that doesn't mean it is easy for a club player (that's sort of the point of writing a book aimed at club players).

Rather than just bitch, what can I do? In practice, I make my own little database by playing as many blitz games as I can in the opening. For instance, recently I started playing the Sicilian as black so I've been saving each Sicilian game in its own file (after game ends, go to Game || Save PGN in ICC menu) so I can get a better sense for what I see in practice in games I play against real people, and helps guide my focus if I feel like reading annotated games in the Sicilian, or opening books, etc..

32 Comments:

Blogger Tommyg said...

That is just one of the reasons I like playing at chess.com, either correspondence or real time! They have an opening explorer where you can check out the openings in your and other's games. You can see how often certain moves or lines or being played AND the eventual results!

That service has helped me to sort of finally kind of get a grasp on opening study. :)

I am rather bullish on Chess.com!

2/22/2010 01:10:00 AM  
Blogger Blue Devil Knight said...

Tommyg: ICC has an opening explorer too. Does the one at chess.com let you narrow down the explorer to games within a certain rating range?

2/22/2010 01:16:00 AM  
Blogger Tommyg said...

That they do not have. The best they have is that you CAN search any other member's games. But an opening explorer by opening range they don't have. That is a great idea.

In fact I just did an online database search, and a search at chessgames.com and neither of them allow one to do an opening search by rating! You may be on to something here.

Although the way I use an opening search at any of the two site or using chessbase is that I never really search past the first 5 moves or so since that is all I need to be concerned with. The only time I go deeper is when I am analyzing a game of one of the masters and that is just because the material is there.

But I think you are right when you say chess authors should look at that kind of thing when writing "starting out" books etc. etc.

2/22/2010 01:38:00 AM  
Blogger Paul McGaugh said...

As much as ICC charges it would be nice if they had this feature. I think chess.com is the way to go. ICC has become extremely overpriced. It seems they keep raising the price there. I first joined ICC in 2000 and have remained a member since. If I remember correctly the price for joining a year membership in 2000 was $20-$25 dollars. Now it's almost $70.00 dollars for a year membership. The extra features that ICC offers isn't worth the extra money to me although I do like GM Alterman's Gambit lectures. Now I play on the free servers to include: ChessJam, ChessCube and FICIS and I might add without the rudeness found on ICC. Cheers

2/22/2010 11:17:00 AM  
Blogger chry said...

I think it won't be that helpful because many of the lower rated players (like me) lose their games becouse of other blunders and the opening isn't that intersting.

2/22/2010 11:40:00 AM  
Blogger Blue Devil Knight said...

chry: the point is to see the openings' relative frequencies, not the ultimate conclusion (which for all blitz games would suck).

Of course they'd have to filter the database, not just let every game in. This is easy to automate. Then, let users filter the games more (e.g., show me the stats for players between rating X and Y, or within Z of my rating).

If you do the math: number of members times dues per year, it seems ICC makes well over a million a year in dues from members. They do provide some useful lectures now, but something like this would be extremely useful and would put ICC back in top over chess.com.

2/22/2010 12:04:00 PM  
Anonymous Anonymous said...

Whats your icc name?

2/22/2010 06:04:00 PM  
Blogger Tommyg said...

So I have been thinking about this (and granted these are a patzer's thoughts!) but it makes sense to know the percentage of what is played at the lower levels so we can prepare for that. BUT I think that it also makes sense to study the so-called "best" moves that the better players play. Not as deep into a line as they do for there are other things to study. :)

I want my music students to listen to the best players! How deeply they listen (read: transcribe that player's playing vs. just listening) depends on the students technical level and understanding. But I do want them checking out the best players from the get go. So as my students improve their technique (tactics), and get some performance experience (OTB long games) I gradually increase the depth of their listening and transcription, (diving deeper into theory, game analysis and strategy). But always listening to the best from the get go.

Have I gone off topic?? :)

2/22/2010 10:46:00 PM  
Blogger Blue Devil Knight said...

TommyG a good analogy, I'd just refine it to make it applicable to my argument.

Say you teach your students only a biased subset of the best play. E.g., you leave out 50% of the chords (or whatever it is you guys learn) that they will actually need in practice, but on the other 50% you teach them really well. Would that be good teaching? (OTOH at least in chess there are general principles we can follow in the opening).

2/23/2010 12:28:00 AM  
Blogger Blue Devil Knight said...

The more I think about this, the more obvious it should be done. We don't live in an age where it is expensive to store games, where you don't want to just store any old piece of shit game.

But now they have their hands on a huge amount of data about opening trends as a function of playing strength. That's just valuable data for those that want to teach chess and those that want to learn chess.

People could still be locked into the old 'data is expensive' way of thinking, where you had to be all worried about having a gig of data. Now that is just not the case.

For instance, from their database of millions of games, you can explore opening variations extremely quickly at ChessOK at their website, for free.

Come on ICC, move into this millenium! Provide the data, let us show you how useful it is!

Seriously I'm about to move over to FICS or chess.com. F it.

Anon: it's a secret I gave it out once and got stalkers so now I don't give it out here. I let that one expire now I have a secret one named bababooey!

2/23/2010 01:24:00 AM  
Blogger Tommyg said...

Hmmm? Now you've got me thinking!


Maybe my analogy was a bit off when I discussed listening habits. Listening habits in music would relate to just playing over master games in chess (which is a lot like listening to records)

So I need to see how this relates to music theory because openings are just that: theory. There are principles but theory goes in and out of fashion.

In music, theory is just a way of codifying stuff so we can name it and then hopefully make better use of it. IE: The building block of music theory (as taught in the colleges) is four part voice leading based on the rules of Bach. However, Bach did not know they were rules. He was basically making the stuff up as he went along, based on what had come before him.

Much like chess opening theory, music theory is ever evolving and going in and out of fashion. For instance, there is an interval called the tri-tone which in the days of yore was frowned upon and called the devil in music. Nowadays it is VERY accepted, and commonplace.

BUT you can also have a modern composition that doesn't have any tri-tones in it at all. More to follow:

2/23/2010 01:46:00 AM  
Blogger Tommyg said...

To continue:

Music theory has evolved to the point where the rules are more like good solid principles. Knowing them will help you name it so people can perform your music better.

And of course there are all kind of arguments as to what makes for better harmony in music. Just like there are endless arguments about what makes for better opening play in Chess. The Scotch opening was dead until Kasparov revived it. Modal harmony in jazz was NOT commonplace until Miles Davis recorded Kind of Blue. Then there are the chess players who like to create anarchy at the board regardless if it is theoretically sound or not. Ornette Coleman (and others) did the same in Jazz. BUT here is the thing: Davis knew his standard jazz harmony down cold before he made Kind of Blue. Ornette Coleman knew be-bop down cold before he broke the rules.

As a music teacher I want my students to eventually be themselves both as performers and composers. TO do that one does (generally) have to study the greats which by it's very nature does include studying theory and it's evolution.

Which brings me back to studying the great players and their opening moves in the openings I have selected.

If I have a trumpet player that wants to be a great jazz player. I want him or her to study all the great jazz trumpet players from Louis Armstrong to today.

But here is where we merge a bit!!

I don't want them to STUDY say Clifford Brown before STUDYING Armstrong. They can listen to both but they need to study Armstrong first.

That is why I mentioned in an earlier reply that I want to study the best moves but I don't want to get bogged down in the theory (ie: 20 moves in the Marshall attack)

I don't need the Marshall attack..I just need to know the first say 6 or 7 best moves in the Ruy. And then go deeper as my ability develops.

So I think a database based on a ratings range would be GREAT for practical planning, but for overall edification and improvement I think studying the "best" moves from the best players would lead to better overall chess development.

2/23/2010 02:07:00 AM  
Blogger chesstiger said...

Granted that i am not someone who is into openings or even considers to study theory but i find all this fuss about who or which rating group plays which kind of opening a bit over the hill.

At our level, the patzer group (yes, i am still a patzer), every opening is good, even those out of fashion in GM play, if one knows/learns the plan behind these opening moves so that one can figure out how to take advantage if the opponent goes out of theory on move 3 or 5 or ... .

2/23/2010 03:47:00 AM  
Blogger Blue Devil Knight said...

Chesstiger: yes, good point. However, in the context of an opening book it becomes important which lines are examined.

Tommyg I can do this no matter what you say though: take whatever you think should absolutely be learned about X. Now randomly select half of that material and teach it to your students. That's effectively what is done in most opening books or opening databases that don't accurately reflect what we patzers will see.

As I already said in my original post (this relates to chesstigers comment too) there are obviously principles we can use, but when the goal is to learn what you will see in practice, or to teach others about what they will need in practice, the present static and biased databases are weak.

There's a huge mine of cool data that is not being collected. It should be available, especially from ICC since we pay them so much each year. They can easily afford the fifty grand or whatever it would cost to provide this service. As I said, just provide the data I will show you how useful it is. It would spawn great new trends in opening books written for real players, where they could really take the time to explain why this unprincipled move is bad, show the refutation rather than say nothing. It would allow for a great advance pedagogically over those that use IM/GM databases for ideas.

It'd all be in the spirit of Euwe's great book 'Chess master vs chess amateur.'

Anyway, my argument seems fairly trivial. The data could only be helpful. Someone who was uninterested could ignore it. But the GMs writing for beginner/club players would soon realize how useful it would be to have such a treasure chest of data. As would the lowly club player just starting to futz around with a repertoire, trying to figure out how much time to spend on various lines.

2/23/2010 11:27:00 AM  
Blogger Grandpatzer said...

I wonder if we could start an online petition, and if that would have any effect.

I got sidetracked, but I was creating an opening database where I take all my 8000-odd games and catalog every book line played. Not as daunting as it sounds, considering how early we deviate.

2/23/2010 12:16:00 PM  
Blogger Blue Devil Knight said...

GP or at least an email campaign. I tried unsuccessfully to find contact information for the CEO and such of ICC last night, didn't find it on their web site.

2/23/2010 02:55:00 PM  
Blogger LinuxGuy said...

Openings are important, but Kasparov playing the Scotch didn't affect my choice too much, but perhaps subconsciously a little.

I simply didn't want to play the Ruy with locked pawns, ad nauseum, as White.

Some openings really are easier for beginners. Closed openings can be the toughest because it's not so clear what to try.

If they told you which ones were the easiest, a lot of people would probably copy those.

2/23/2010 03:16:00 PM  
Blogger Jeff said...

Analysis of results broken down by opening, color, and rating class would be interesting. Is the "King's Gambit" more effective at class C than class B for example? And how does black fare with the French Defense as a class C player versus as a master.

I'm a data guy though. I love data.

2/23/2010 03:36:00 PM  
Blogger Signalman said...

What command is it at ICC to obtain this information ( statistics about the opening )?

I have seen somewhere that games are saved at ICC, but I think it applies only to high rated players. Can't remember where I saw that now.

I have to say, that I have experienced far less rudeness ( & indeed foul and abusive language ) at ICC that I did at FICS for example, particularly from guests.

I have only been on ICC since January, but so far a very pleasant experience. I also thought the cost was reasonable.

Chess.com I haven't ventured into playing at, just reading articles and fora. Many comments suggest that playing strength is quite low, and I thought it was only correspondence-style,which I play at other more-dedicated sites. I think that many of the "articles" there are very poor ( unless provided by GMs & co : Silman's are invariably excellent) and some of the comments that I see, particularly on those articles from WGMs are bizarre indeed. I doubt if some of these are drawn by chess motives at all !

FICS ( or rather an associated site ) does have a searchable ( & email-enabled ) database of all games played there, which is also searchable by rating, if I recall correctly. Maybe that is what you want to view ?

I have heard almost nothing of playchess.com (which is also only 30 Euros a year - a bargain ), but did try it as a guest and enjoyed it. I also wasn't sure if there were any tournaments/leagues there, which was the pull of ICC: T4545 League and the regular standard tournaments. !

I recently played on ICC in the Monthly standard 60,0 and enjoyed the games a lot : just what I need !

Good to see your blog regularly active again. It was one of the first I started to read when I returned to chess a few years ago.

2/24/2010 03:19:00 PM  
Blogger Signalman said...

I should that from my standard games on ICC, as White playing e4, I am most likely to meet the Sicilian (Dragon or Scheveningen )the Scandinavian and the despicable French.

As Black, 80% of games so far start e4, and about 40% are King's Gambits, followed by 2Nf3 etc. I generally try to avoid the Spanish, if possible.

2/24/2010 03:43:00 PM  
Anonymous Anonymous said...

so BDK, if you had 150-200 games in the Marshall Attack of the RL in which White was not a strong player, what would you do with them? what would you want to do with these games?

2/24/2010 04:01:00 PM  
Blogger Tommyg said...

Might this be what you are looking for?

http://openingmaster.com/OM-Portal.html

I haven't really checked it but it MIGHT be relevant to what you are looking for.

2/24/2010 07:32:00 PM  
Blogger Blue Devil Knight said...

Signalman: 'finger openingsbot' at ICC should explain it. It's not exactly an opening explorer, but does give you statistics for particular moves.

Is it true that FICS has this feature, that it lets you download games from any rating? Can anyone verify?

Anon: For opening X, I'd like to look at the relative frequency of different moves made. This would help me know what to think about and prepare for.

Tommyg: Not sure, that's a pretty small database and my hunch is it's from titled players.

We can't blame people up to the 1990s for only saving titled players' games. Saving them in paper, or even on computers, would be a huge waste of space. But now that memory is cheap, there are new vistas opened up for chess databases.

2/25/2010 12:22:00 PM  
Blogger Signalman said...

See http://www.ficsgames.com/

Has FICS games, but not "FICS" itself....

I often use it, but still, I want to know what the "Best" replies are, regardless of what us amateurs play.

Half the 'fun' is working out why the best moves are the best, then finding the refutation ( or not ) of the moves we actually play in amateur OTB/Internet.

2/25/2010 04:54:00 PM  
Blogger Signalman said...

..and the link from Tommyg is interesting, but you have to pay for the real thing, and , as you suggest, and I suspect, these games won't be from evry Tom, Dick or Harry.

Also of interest are the internet correspondence sites ( IECG etc ) which also have downloadable & searchable databases. These are a mixture of true perfection-seeking correspondence games, plus the usual mistakes !

2/25/2010 04:57:00 PM  
Anonymous Anonymous said...

Hey guys, very interesting discussion. My experience on ICC (where I'm in the 1500 range and mainly play 15 0 games) is that as a 1...e5 player I mainly see the Italian game over and over. Some King's Gambits, an occasional Scotch (or Scotch Gambit) and when I get the Ruy it is usually the exchange variation. I would love to see a breakdown of openings played by ratings class as it helps you to know where to focus your study time.

2/25/2010 05:45:00 PM  
Blogger Blue Devil Knight said...

Signalman: great find!!! That's exactly what I'm talking about.

Time to look into switching over to FICS.

2/26/2010 10:57:00 AM  
Anonymous Steve Eddins said...

BDK, you said in a comment that you once had a problem with "stalkers" on ICC. Can you tell us more about what happened? I've never been very happy with the use of anonymous handles online, and I was thinking about switching to a new handle based on my own name. But I'd like to hear about your experience before I decide.

3/02/2010 07:36:00 PM  
Blogger Blue Devil Knight said...

Steve: I was exhaggerating a bit. Once I mentioned it a lot of people started observing, wanting to play me, commenting etc.. I found it stressful and distracting rather than fun and comraderie-generating.

3/03/2010 11:14:00 AM  
Blogger transformation said...

chessbase will tell you all your stats on each opening, worth perhaps more than stats for ALL players. a good investment, or cblite instead.

i have 16,000 of my own games and believe me, i can harvest my own activity to see what works and what doesnt, and compare to a 4.2M game Megabase.

3/09/2010 04:48:00 PM  
Anonymous Adam Rinkleff said...

You should try FICS

Freechess.org has the exact same code as ICC, except its free. FICS was created when ICC started charging, because a lot of people saw no reason to try and make money. FICS is great, and its free, and it has free stats (unlike chess.com, which you have to pay for).

6/20/2010 11:45:00 PM  
Blogger Blue Devil Knight said...

Adam, yes I'm now a happy FICS user. I never used the services at ICC anyway, so have nothing to miss there. The only difference is slightly fewer people to play, and people aren't quite as good (e.g., my blitz rating at FICS is about 100 points higher than at ICC). However, none of that matters I just play casually now to help me relax.

6/21/2010 10:36:00 AM  

Post a Comment

<< Home