Is Tom Izzo really the best tournament coach ever? Well...maybe. A look at subjective value assessments in real world analysis.

Statistical analysis is a tricky thing. The most common misconception (in my opinion) is that real world analysis is similar to an academic math problem. Most pre-graduate math problems have a clear structure, pre-defined assumptions, and most importantly, only ONE answer (makes grading easy of course). That reflects poorly on pre-graduate level math classes but that is another topic.

Coming from this background, it is easy to consider any piece of numerical analysis as a math problem that the analyst worked out behind the scenes and is now telling you the answer. In fact, this perception is commonly (and amorally) cultivated by the analysts themselves. This is a problem even when the analysis is done well. Real life problems almost always require layers of assumptions and perspectives to give the best possible answer. Subjective value assessments are unavoidably an integral part of analysis.

Take this article from fivethirtyeight.com which attempts to grade college basketball coaches on their NCAA tournament performance. First off this is a good piece of analysis (Ha! Everyone who knows I am a Michigan fan thought I was going to trash this article for lifting up Izzo as the best tournament coach). I do, however, take issue with the fact that the article claims a conclusion to a very complex and difficult to quantify matter (coaching ability) based on this one single metric, wins above expectation (WAE). Even a good metric needs context and alternate perspectives.

Taking a look at his data, it is readily apparent that the author simply added up every coaches' WAE across their entire career. The exceedingly obvious next step (not done in the article) is to divide that number by the number of tournaments each coach has participated in, thus producing a WAE per tournament appearance. For example, Tom Izzo has a career 14.6 WAE and 18 appearances, producing a WAE per appearance of 0.81. I grabbed a few coaches (not randomly, for the purpose of meaningful comparisons) from that list and calculated their WAE per appearance for context:

Brad Stevens: 1.62

Tom Izzo: 0.81

John Beilein: 0.77

Mike Krzyzewski: 0.14

Look! Tom Izzo is still really good. For those of you convinced I am working to undermine that conclusion you are going to be disappointed. I chose these coaches to make a few points. The number of tournament appearances matters. Tom Izzo and John Beilein have a similar WAE per appearance, but Tom Izzo has double (18 versus 9) the appearances. Assuming Beilein coaches many more years (please please please let this happen) and makes 18 tournaments, it is entirely reasonable to expect that he ends up with a similar WAE as Tom Izzo since he is already on pace to do so.

Another comparison is Mike Krzyzewski, who is a clear notch below the others on this list. The difference here is that he has four titles titles compared to one for Izzo and zero for the others. This sort of analysis treats all wins the same even though you could very easily conclude that a coach who wins it all one year then bombs out early the next year is better than a coach who hits expectation exactly two years in a row, depending on your perspective. Another factor that I won't go into too much detail here is that I think this metric is unfairly punitive to the higher seeds. Let's just put it this way: this metric determines that the average five seed winning two games (0.9 WAE) is a better coaching performance than the average one seed going to the final four (0.7 WAE). I think there is some gray area around that determination.

My final illustration is Brad Stevens, who in five tourney appearances with Butler produced 8.1 WAE, good for 1.62 WAE per appearance. When looking at WAE on a per appearance basis, it is Brad Stevens rather than Izzo who is the best tourney coach (-by far, as if that qualification was necessary in the original article. We have clearly shown that it isn't that simple). The illustration is not necessarily that Stevens is better than Izzo, but that it is difficult to determine the value of Izzo's long term consistently high performance versus the extraordinary short term performance of Stevens. It is a subjective comparison with no clear right answer.

And this brings me to the point I wanted to make all along. It is exceedingly tempting to eliminate the messy subjective comparisons by making your own determination and simply presenting your conclusion as if it is the "right" answer. I mean, when you do that you get to write click worthy titles like, "Izzo is the best tourney coach ever - by far" instead of the more accurate "Izzo is either the best ever or just one of the best ever depending on how you value his long term consistency versus Brad Stevens' remarkable five year run." And yet, the second is a better answer than the first.

As an analyst you want your work to be noticed and appreciated. Nuanced analysis tends to be ignored because most people don't want to take the time to understand all the underlying factors. This leads to either shoddy work (not the case of the above article in my opinion) or to the analyst making the subjective value determinations for you without telling you that he has done so (bingo). I think the author of the article in question simply decided that Izzo performing at a high level over 18 appearances made him a better coach than Stevens, who was only in five tournaments and would most likely regress to the mean in future years. Instead of telling you he did that, though, he just avoided the matter entirely. 

I think it is important for 1) analysts to be aware of this temptation and try their best to present the issue at hand in as fair and complete way as possible and 2) for consumers to always try to dig beneath the surface and make sure you agree with the assumptions in place before accepting the conclusions being presented.

On Professional Starcraft, Koreans, and Hard Work

Sometime during my Junior year in college I happened to be waiting for a meeting to start.  I decided to kill time in a computer lounge near the meeting location.  The computers had the video game starcraft already installed (best university computer lounge ever?).  It was a game I had played when I was much younger, probably 12 or so.  I mostly watched my older brother play, and I had gotten about halfway through the single player portion of the game before getting bored and moving on to another game. This time around, though, my interest was sparked and I played for about 10 minutes until the meeting started.  That simple event sparked a hobby that I still enjoy to this day.

The game itself is easiest to explain as chess played in real time (no turns).  You collect resources, build fighting units, and go smash the other guy while he does the same thing.  In addition to strategy, it takes a lot of finger speed and concentration to play the game well.  My younger brother would find great amusement watching me bounce around in my chair as I played.  I have tried to be more relaxed before, but every time I get into a close game, my competitive side takes over.  I tense my shoulders, my face gets closer to the screen, and my legs rapidly bounce like I am getting ready to sprint.  My wife knows to not tell me anything I should be remember while I am playing, because odds are I will either not remember or not hear her in the first place.

One thing that really draws me to Starcraft is the difficulty in playing it well (I don't actually play it well, relative to people good at the game, but compared to your average non-gamer I would seem unbeatable).  You have to put in endless repetitions to develop the muscle memory, game knowledge, and reaction speed needed to be a top player.  It is a little bit like tennis in that regard; if you want to be good there is simply no replacement for putting in massive amounts of time playing the game. Take a look at this video (just the first half) which gives a few brief shots of a pro's keyboard as he plays. 

You know how a really clean golf swing makes old guys all giddy? Same thing when a Starcraft player gets to see a pro's keyboard when he plays.  Actually that is just a guess, I have no idea if golfers get giddy about a clean swing.  But nerds definitely get giddy over NaDa (one of the best ever) playing Starcraft.

Another fantastic aspect of starcraft is that just about anybody can be good at it if they so desire.  In any sport there is a combination of practice and natural ability that leads to being good.  Lebron James is good at basketball because he spends a lot of time dribbling, shooting, passing... oh and he is a 6'8", 250 lb. statistical anomaly of a human being.  I could have spent my whole life practicing basketball and I would never be as good as James, or even as good as he was in high school.  I don't mean to suggest there is no natural skill required to be good at starcraft.  This is something commonly debated amongst gamers, actually.  Regardless of your opinion, pretty much everyone agrees that the genetics factor, so to speak, is far less important than your typical sport.

When I was first getting into Starcraft 1 (more accurately, Starcraft: Broodwar), I poked around online and discovered a thriving professional scene based almost entirely in Korea. The Koreans had individual and team leagues, team houses with coaches and maids and players who trained up to 12 hours a day.  There were occasional attempts by top foreigners (all non Koreans are called foreigners when talking about starcraft, interestingly) to break into the scene, but none of them found any lasting success.  Every year an event called WCG (World Cyber Games) matched up the top three qualifiers from every country who wanted to participate.  In Starcraft 1 Koreans took first every time.  Here is an interview question directed to one of the Korean participants in the 2010 WCG:

         Q: How strong were the foreigners compared to last year?
         A: The same, not much improvement. As long as I didn’t play too loosely, I could win easily.

In 2010 Starcraft 2 came out, which was hailed as the big chance for the rest of the world to finally catch up with the Koreans.  There were a number of factors that made it the perfect opportunity, I will list these in bullet points because I am a math guy and I really like bullet points for some reason.

• Starcraft 2 is easier to play than Starcraft 1.  The blazing speed I talked about earlier is still needed to be good, but not quite as much as Starcraft 1.
• Most of the top Starcraft 1 Korean players didn't switch to the new game, meaning a big chunk of Korean talent and sponsor money weren't a factor.
• The excitement of the new game injected a lot of money (sponsors) into the scene, so foreign teams were able to start their own team houses, and even sent many of their top players to Korea to train against the best.

 And I could go on.  The above is hardly an exhaustive list. The point being that the advantages of the Korean professional scene dropped mightily with the release of starcraft 2, in my personal opinion there was almost no advantage, though it is impossible to say definitely.

You have probably already guessed where I am going with this.  The Koreans are still the best at Starcraft 2, and not by a little bit. In the most recent major tournament (MLG Spring Arena Anaheim, so this was played in the US) Koreans took 1st, 2nd, and 3rd.  They took 17 of the top 20 places overall.  The funny part?  It was considered a good tournament for foreigners.  Now, I am making it sounds worse than it really was (based on seeding and tournament structure it was easier for many of the Koreans to get a high finish, though those seeds were earned) still, it shows the massive disparity.  Overall, the gap has closed considerably, and top foreigners are able to beat the top Koreans from time to time, though not with any regularity.  Still, trust me when I say the Koreans are simply better.

This all brings me to the question, why are the Koreans so much better? For the purpose of time, I will rule out the possibility that Koreans are a genetically engineered video game super race designed by aliens to take over the world once everything is run by computers (though I am still mulling that one over).

My personal belief is that the answer is not found in inherent advantages in the scene or the competition or genetics, but that it actually goes all the way down to cultural values. 

This is a massive oversimplification, but here goes.  Korean culture values and cultivates hard work more than Western culture.  Based on my observation, Western Culture tends to take a "you either have it or you don't" approach.  If you are smart, you excel at school and get into a university, if you aren't, well there is community college and you will be just fine (no offense at all towards community college, I think that can be a great option, just illustrating a point).  In Korean and most Asian cultures, if you are smart you excel and get into a university, if you aren't, then you study your butt off and get into university.  Once again, massive oversimplification but I think most would agree with this point, at least to an extent.

This way of thinking translates into many parts of life.  Culture has a way of affecting almost every aspect of your life.  When it comes to starcraft, I think the Koreans just work harder than the foreigners, simple as that.  I talked earlier about how starcraft is such an involved and technical game that every bit of practice you put into it makes you better.  A foreign player might play 8 hours a day, 6 days a week and call it good.  After all you just worked more than a typical full time job, that is a lot of time put into the game, don't you think?  Yes, it is a heck of a lot of time.  Doesn't mean the Korean playing 10 hours a day 7 days a week won't wipe the floor with you.

The realization that good ol' hard work can be the driving factor of your success is both humbling and freeing.  I am an actuary, which is a profession where you take exams which you pass and get raises, promotions, new responsibilities, etc.  Or in my case I mostly take exams and fail them.  I find after a failed exam that it can be easy fall into the mindset of, "maybe I am just not smart enough for this."  Who knows,  maybe that is true.  Doesn't mean the guy with a similar IQ who studied twice as many hours won't pass the exam while you mope about not being smart enough.

It is humbling because you also have to accept that when you failed it was probably because you didn't put in the time and effort.  It is freeing, because so many obstacles that once looked impenetrable open up to you once you realize the limiting factor is usually your willpower rather than circumstances.