Saturday, January 03, 2009

Loaded Dice and NHL Penalty Killers

The two posts below are interesting, I don't mean to steal Showerhead's thunder here, but rather to enforce the merit of the direction that he is heading with this PK stuff. I know that a great many readers of the Oilogosphere do not view the comments sections, and while that is generally wise, you should make an exception for those posts.

The issue of shot quality always rears it's head around these parts, more specifically the influence of individual players on the shot quality on their own goalie. I'm sure it is obvious to most that this is usually because people who live in the moment a lot ... well they need something to cling to, bless them. And while I think that everyone agrees that there is an impact, there is a wide disparity between the magnitude of this in the views of people like Tyler Dellow and people like Robin Brownlee.

Below is a crude model of dice rollers and the Edmonton Oiler PKers. We all know that, given 180 rolls of a die, we expect to see a '6' rolled thirty times. We also know that it's damn unlikely that 10 dice rollers will all throw a '6' exactly thirty times. Shit happens. Some guys will run hot and others cold. If you played board games as a kid, you know this already.

So the question is; how much of the save% behind individual PKers is just coincidence? How much (the remainder) can be chalked up to their ability to affect shot quality?

The methodology:

Take every Oiler defensemen or forward who has been on the ice for 60 or more shots against (a cool dozen Oiler skaters as it happens) and pit them in a game of 'rolling for sixes' with weighted dice, or 'Vic Sixes' as the kids are calling it now. The dice are weighted the same for everyone, the chances of rolling a six are 16.08%, the same as the chances of an opponent scoring on a shot during an Oilers 5v4 PK this season so far (and that sucks by the way, but history is telling us this has no sustain, in fact it's screaming it in our ear).

So, by way of example, Souray has been 5v4 PKing while the opponents have shot 123 pucks on the Oiler net. So he gets 123 rolls of the dice, weighted as above. Same goes for every other bugger. And all of the Oilers play 1000 games of 'Vic Sixes (TM)'.

And if we look at the results of Vic Sixes - Game One - then Sheldon may have done great or done poorly, depending on the love he got from the dice on the day. In this case he got 21 'sixes', the same number as goals against that he has been on the ice for at 5v4 PK. Is he crap at Vic-Sixes? Is he unlucky at Vic-Sixes? I mean the chances of this happening by coincidence alone are only one in three, surely he just sucks at Vic-Sixes (I know I won't be picking him for my team tomorrow!). I mean nobody is just that unlucky, are they?

Turns out that they are, in the right measure. Simple math tells us that one in three are just that unlucky. No more and no less.

If you have twelve players then one guy should be in the "there is only a one in twelve chance he is just unlucky" range. One hot roller should be in the "there is only a one in twelve chance he is just lucky" range. Two hot rollers should be in the "there is only a one in six chance they are just lucky" range. And so on, because the world is round.

Now let's load the dice, if we know that Cole has stunning PK numbers at 5v4 (he does, best on the Oilers), why isn't MacTavish playing him more? Is Craig really that MacStupid (Gregor callers nod in unison), or does he somehow, magically, know that it is just luck. That can't be, can it? I mean gosh darnit, surely his staff is not counting scoring chances.

How many times has Robin Brownlee told us that there is no such thing as luck, this as he posts from Joanne Ireland's basement while smoking left handed cigarettes with Terry Jones. Surely you are as proud as me that he called bullshit on Horcoff and Moreau, no? Wake up and smell the Brownlee, motherfuckers, there's no such thing as luck, at least not when you're giving 110%, that's for damn sure. And when you're giving 115% (near the theoretical limit, btw) well there just isn't any luck at all. Don't listen to NHL coaches, they're just blowing smoke up your ass, thank Christ we have folks like Mark Spector to parse that information into meaningful bits for us.

Anyhoo, turns out that if you think in these terms, pretend for the fuck of it that NHL coaches aren't complete fucktards and that NHL beat writers just might be ... and plot the results out assuming it was just luck. Well you'd expect the dice rollers results to be linear with expectation, and they average a correlation of .971 to linear (by the by, Oilers PKers are at .976). And you'd expect the standard deviation to be .29, it is in fact .287 (by the by, Oilers PKers are at .297).

Now let's load the dice. So we know that Cole is the bomb and Moreau is the suck at rolling them bones this year. Let's believe it's not in the dice, but rather in the wrist action (OilFans.commers nod in unison). We'll give the extreme guys the benefit of the doubt, a 0.1% better chance of rolling a six, that's not much folks.

Visually, which is the way the human brain really sees numbers, and with dice loaded to the extremes (-.005 represents the worst dice roller being responsible for 0.5% (1 part in 200) of his bad luck being the consequence of poor wrist action and a lack of positive thinking.

By average correlation to linear and average spread (standard deviation):

By rank correlation to linear and rank spread (standard deviation):


Blogger mc79hockey said...

Vic - I'm still reading the post but you don't get enough tips of the cap for the quality of your writing. The bit about Brownlee in Ireland's basement smoking left handed cigarettes with Jones was fantastic.

1/03/2009 5:10 pm  
Blogger Vic Ferrari said...

I grew tired of writing before I explained what those graphs really are.

The bottom axis, on both graphs, is an artificial weighting of the dice. So if you go to .010 ... this is the result if you credit the PKer with the best save% behind him (Cole) for being responsible for .010 of that, and the rest being dumb luck. And the worst (Moreau) being responsible for .010 of the badness.

Coles has a .957 save% behind him btw, Moreau a .777.

And everyone else gets a proportional change based on rank. .0082 heavier dice for the second ranked guy by on-ice save% (Pisani) Just .001 change in dice weighting for the 6th ranked guy, etc.

The low point on each graph is the point at which the dice weighting most closely matches the results of a game of Vic-Sixes. (i.e. random results).

Four different ways of looking at it, giving estimates of about, -.001, .003, .003 and .005.

So while it is extremely likely that players ARE INDEED affecting shot quality ... at the extremes the effect should result in an extra goal against every 3 or 4 seasons because of that.

The shots data that Showerhead is looking at below is far, far more important.

1/03/2009 7:22 pm  
Blogger Vic Ferrari said...

And just to add, since it is remarkably easy to do by shifting the data over one row in the spreadsheet:

The effect of skaters on PP shooting percentage, for the Oilers this year, is enormous. In the .045 to .080 range in terms of impact on shooting%. That's a whack, folks.

1/03/2009 7:53 pm  
Blogger Showerhead said...

Sunday seems to be ask Vic questions day in my world.

My question is that if you start by weighting your dice with the opponents' shooting percentage while Edmonton is shorthanded... then in the end are you just measuring how consistent that shooting percentage is regardless of which Oiler penalty killers are on the ice? Saying that given X%, the number of goals against each Oiler pk-er is matching closely to the amount that "should" happen according to statistics?

On one hand, IF players do have an impact on the save percentage behind them, you might expect that the opposition's shooting percentage could theoretically vary quite a bit depending on which Oiler pk-er is on the ice, no? You might expect to see more variation than the stats would suggest, and if you did there would be some evidence there?

On the other, it is a common thread that the differences between great and average in the NHL are relatively small. A "great" goaltender such as Roberto Luongo has a sv% just 1.075 times greater than an "average" goaltender like Ty Conklin. Perhaps variation from individual to individual on save percentage would not leap off the page IF it existed.

Especially IF (yes the capitals are getting ridiculous now, I'm being theoretical) those same players were all playing the same crappy pk system, 4 of them on the ice at all times, and their collective incompetence or proficiency combined to skew the shooting percentage just that little bit.

From a different angle, let's say that we know what Dwayne Roloson's expected pk save percentage is, based on how many years of past performance you think is relevant. Depending on how much data you have or have confidence in, that expected save percentage would have a spread for what could rationally be explained via normal variation. And then you'd have potential data points that are simply outside that curve. What IF Roloson or Garon posted a season whose numbers were that far out? Outside the tight curve of what's expected AND outside the fatter curve of what could be explained by normal variation/luck?

This could be a case of the eyes fooling the brain but this seems in my mind to be the sort of thing that would have to be come at from a number of different angles before one or another view was proven. Thoughts?

1/04/2009 1:10 pm  
Blogger RiversQ said...

Great stuff Vic.

This bit jumped out at me:

Now let's load the dice, if we know that Cole has stunning PK numbers at 5v4 (he does, best on the Oilers), why isn't MacTavish playing him more? Is Craig really that MacStupid (Gregor callers nod in unison), or does he somehow, magically, know that it is just luck. That can't be, can it? I mean gosh darnit, surely his staff is not counting scoring chances.

Funny enough, I looked at this almost exactly over the summer and never posted it.

I looked at the NW division players for the past couple of years and compared their 4V5 GA/hr rates with their share of team PK time and there is nothing there whatsoever. In fact for 4 out of the 5 teams (I think Van was the exception), the trend was actually negative. A decent whack of the guys that played the most had average or downright terrible GA rates.

So your choice of explanation is:

1. MacT's mindnumbing game management has managed to telepathically infect his NW counterparts.


2. Coaches know something that isn't fully represented by PK GA rates.

I think everyone appreciates a smell test for this stuff now and then. Whether that's checking the opposition with matchups to Zetterberg and Thornton or just looking at how NW coaches dole out the ice time, it's always important IMHO.

I think there's a heap of luck involved in individual PK performance without a doubt, and certainly some other muddying factors. Personally I do wonder about the faceoff personnel on the PK. Just like there are the Horcoffs which have it tough, I think there are also the Martin St. Louis-types who don't hop over the boards until the puck is in hand or in the other end of the rink. These guys have some nice PK numbers, but they are only penalty killers in a narrow light.

1/05/2009 12:00 am  

Post a Comment

<< Home