Tuesday, December 11, 2007

Shit Happens

Yesterday I read this excellent post at sisu hockey, and it's moved me to chip in on the subject of plain old, unglamourous luck.

There is some grooviness surrounding save percentage at even strength, so today I'll take a couple of kicks at that, testing NHL players VS dice rollers. I ran this data a week ago, using 5v5 and 4v4 numbers only, and only when the goalies for both teams were on the ice. It's repeatable using a cut and paste of data from Desjardins' advanced hockey stats site, if you're a dab hand with Excel it should only take half an hour or so to do this sort of thing.

What I did with the NHL players:
  • Looked at a player, say Cogliano of the Oilers, and noted that he'd been on the ice at even strength for 13 goals against, and 148 shots against. That's a .912 EVsave% when he's been out there.
  • The Oilers on the whole, at the time, had a team EVsave% of .907 at that point.
  • If Cogliano had seen that .907 EVsave% behind him, he would have been on the ice for 13.8 goals against, round it up to 14.
  • So the expected goals against was 14, and the actual was 13. So Rooster Cogliano gets a -1 Diff-number from me here. Meaning that if we believe that he has no significant factor on the goalie's even strength save percentage, then he had been lucky to to the tune of 1 fewer goals against.
  • Do the same for everyone else in the league.

What I did with the dice rollers:

  • Cogliano gets an imaginary dice roller.
  • This dice roller gets 148 rolls of a die, that's the same number as the shots against that Cogs was on the ice for.
  • The die he uses is weighted so that it rolls one thru five 90.7% of the time, because that was the EVsave% of the Oilers overall. This is done using a simple script and a random number generator.
  • Each time the dice roller rolled a six, it counted as a mock goal-against.
  • Repeat for the other 703 NHL skaters who have played a game this season, using their unique personal and team stats of course.

Finishing up:
  • Sum up the total number of players with a -1 Diff Number, like Cogliano.
  • And the number of guys with a -2 Diff-number, and a -3 etc. -10 through +10.
  • Do the same for the dice rollers.
  • Compare dice rollers to NHL players and be confused.
Both lines are going to be a bit bumpy and irregular. There is buggerall that you can do about that for the NHLers, but for the dice rollers you can repeat the exercise 100 times and take an average to smooth things out. So in the first iteration the dice rollers had 111 guys with a -1 Diff-number, like Cogliano did in the NHL world. The next iteration was 98, then 122, and on and on. The average of 100 iterations was 106.4 dice rollers with a -1 Diff-number, which is the difference btween how many sixes they rolled and the number of sixes we would have expected, knowing the weighting of the dice. Capiche?

Anyhow, the pretty picture:

You can click on the image to enlarge it. The blue line is the NHL players, so for example, there were about 80 players with 2 more EV-goals-against than would have been expected if they'd seen their team's average EVsave% behind them.

The pink line is the dice rollers, the average of the 100 seasons worth.

The area between the two is part randomness (we have 100 dice seasons, but only one NHL season after all), and partly the effect of players impacting shot quality as defenders, and partly the effect of the guys who play more against better shooters and play makers. That's a lot of theory competing for a tiny patch of real estate.

NHL players who've found themselves on the right side of this curve are probably having great starts in terms of EV-, at least relative to their teammates and to their previous seasons. The players who've fallen on the left probably aren't getting much love from the fans in their towns right now. Not to worry, it's mostly just shit happening, they're almost certainly going to see a change in fortune in this regard over the rest of the season.

And the yellow line, well that's where the blue line will be in April. The player results will smooth out a bit as the sample gets larger, and drop down and widen out to be tight to the yellow line. It will happen, and there's not a damn thing any of us can do to stop it.


Blogger Jeff J said...

"Not to worry, it's mostly just shit happening, they're almost certainly going to see a change in fortune in this regard over the rest of the season.

Only if the coaching staff is capable of looking beyond the counting numbers.

If they can't, then the player is benched and his UFA year is screwed. In the offseason he signs with the Leafs at a price below what his market value should be. Then he lays a beating on the team that drafted him eight times the following season. Generally speaking.

12/12/2007 8:55 am  
Blogger Vic Ferrari said...

Which player did the Habs lose to the Leafs that way, Jeff? I can't think of any off of the top of my head.

12/12/2007 11:44 am  
Blogger Jeff J said...

It hasn't happened yet, I've just got this horrible feeling that it's going to happen to Ryder. All those Leafs fans on the Rock... Bob Cole...it's inevitable.

Maybe it's Jonas Hoglund deja vu.

12/12/2007 12:18 pm  
Blogger speeds said...

Vic, a question here from someone with vague statistical knowledge.

How differently would the graph look, if any, if ability to out/underperform ES sv% were a skill that a player had, and not strictly the result of "luck". Would one be able to differentiate?

12/13/2007 10:03 am  
Blogger Vic Ferrari said...

speeds, the pink line is the "pure luck" curve. That's just dice rollers.

The blue line is the players actual results.

So to answer you question simply ... there's next to nothing in it if you look at it this way. There would be some separation there if we accounted for the fact that good players are generally playing against good players, but it's minor compared to the ability of players to drive zone time, shots on net, quality scoring chances, and the ability to finish.

Hockey's good that way, perhaps because it's a simple game, the spread of results matches the simple theory. Baseball is a clusterfuck in this regard. I follow links from MC's site quite often, and they always hit the midpoint and miss the spread by a light year, though I have no idea why. The nice thing about baseball is that you don't have to fart around digging up the raw data, it's dead easy to find on the net in a usable format. retrosheet.org, a link from MC or LT I think, is terrific that way.

The exception is something I checked after hearing Joe Torre speak a couple of times during the playoffs 3 or 4 years ago, regarding pitcher/batter matchups ... and if you go LHP vs RHB, or vice versa, actual results vs dice rollers (you can use James' head-to-head equation, it's out of the blue but seems to work, too wide a spread but near enough for this example), anyhow, you get lines that are right on top of each other. There is absolutely nothing to see outside of the noise. Torre is an interesting guy.

And the holds on baseball gamelines are the lowest in the industry, they would have to be larger if monkeys were betting, but the Sabrmetric phenomenon has made the market predictably bad. It's fascinating stuff. A shame that baseball itself is so boring.

12/13/2007 12:41 pm  

Post a Comment

<< Home