2014 NHL Draft: On Quality of Competition for CHL Defensemen

With the Stanley Cup Final already a fading memory and tonight’s NHL Awards giving fans a formal last chance to look back on the 2013-14, hockey eyes are starting to turn to next season and beyond. With the feeding frenzy that is free agency still a week away, the next big opportunity for teams to start to reshape their respective future fortunes comes this weekend in Philadelphia with the Draft.

For the Washington Capitals, it will be new general manager Brian MacLellan‘s first draft and a chance to start really putting his stamp on the franchise’s (presumably new) direction. Armed with newly promoted assistant GM Ross Mahoney and a team of scouts, MacLellan endeavor to re-stock the Caps’ cupboard with future NHLers, snagging a few diamonds in the proverbial rough along the way.

But to get the right “who,” teams have to have the right “how,” and an ever-growing part of that is found in the numbers. And although hockey analytics seems to be growing quite rapidly at this point, it’s still difficult to use numbers to judge prospects. That hasn’t stopped teams willing to put in the work to manually track information, but aside from basic boxscore stats and, sometimes, on-ice goal data, there isn’t much to go off freely available.

Then again, sometimes, basic boxscore information is enough. Rhys J of That’s Offside!, for example, has explored the relationship between draft season points-per-game and NHL games played for defensemen, observing that defensive abilities in junior appear to be overvalued at the draft relative to offensive capabilities (as they sometimes are in the NHL). Defining “scoring” as 0.6 points per game or better in a draft season:

via That’s Offside!

So, in terms of prospect evaluation, we should instantly be suspicious of top teenaged blueliners who lack offense, and maybe consider moving those with plenty of offense up the draft wishlist.

On the advanced stats front, about a year ago, Eric Tulsky observed that on-ice goal information was enough for getting a decent estimate of quality of competition. In the year since, people have started implementing his methods and making the data available to the public. It’s certainly interesting information — but, like for points per game, should decision-makers take it into consideration (more than they already do, at least) when deciding which players to draft?

Methods

(Feel free to skip this section. The most important things to note are that the data are noisy and only available from 2010-11 onward.)

For the 2010-11 to 2013-14 seasons, I calculated ice time quality of competition using the method in the link above, broken down by opposing forwards and opposing defensemen, which you can find listed here, with usage charts for every team in each season here. [Ed. note: This is seriously awesome and well worth checking out now and coming back to later.] The result is something like this:

(Color denotes position – forward or defenseman – and bubble size estimates time on ice; so there’s Caps prospect d-man Madison Bowey playing relatively big minutes against tough competition.)

I’m not completely comfortable using raw numbers here — the estimates are based on head-to-head information from several dozen goals, not hours and hours of ice time information like for the NHL, and, as a result, they’ll overestimate the time on ice for top offensive players and underestimate it for top defensive players by a bit — so I’ll use team ranks (min. 20 GP). (And this seems like a fine place to point out that Extra Skater has a ton of data readily available for this past year in the CHL and McKeen’s has similar data for the past two seasons, so do check those out.)

To maximize the dataset, I tried to include as many games as possible-including preseason, playoffs, and Memorial Cup. I limited the goals in the calculations to even-strength, goalies in. A few games each season had mistakes, which I didn’t bother fixing by hand, being a lot of work for to fix ~2% of the dataset.

I also wanted to somehow quantify each player’s progress and current standing as a prospect. I looked at defensemen drafted in the first three rounds from 2010-13 and assigned each one a “progress score” — just a quick take on the player’s current projection. A five (5) means a top player (Seth Jones, Dougie Hamilton, Derrick Pouliot, Darnell Nurse, and Olli Maatta), a four (4) a very good NHL player (Matthew Dumba, for example), a three (3) a depth NHL player, a two (2) a fringe player, and a one (1), well, worse than that. (Yes, we can quibble about the specific ratings I gave, but changing a few of them up or down a little shouldn’t change the result of the analysis. You may also think I was a little harsh on defensive defensemen, but nowadays, good defensive defensemen need to be able to move the puck — again, effective blueliners in the NHL tend to have been fairly productive in junior.)

Since projections are usually tempered over time, I’ll only compare players to their draft-year peers.

I’ll also focus on defensemen for now, since traditional wisdom says defensive ability is more important in making the NHL for defensemen than for forwards.

Exploring the data

Intuitively, we’d expect top prospect defensemen to be advanced for their age in most facets of the game. If they’re supposed to be able to play defense against NHL forwards in two or three years, they should be reliable against teenagers in their draft season. That should mean their coaches throw them out against the best the opposition has to offer on a nightly basis.

With the exception of Morgan Rielly, that looks true for some of the younger, relatively established defensemen around the league.

And ditto for the top draft prospects out of the CHL in the class of 2014:

But, oddly enough, top prospects coming into their respective drafts frequently weren’t among the top defensemen on their teams in quality of competition, even some top-five picks. That could be a quirk of the small sample size we’re dealing with… but so could the top prospects invariably being on their teams’ shutdown pairings:

Granted, some of these players are playing behind other top prospects who are a year or two more advanced — Seth Jones behind Derrick Pouliot, the eighth overall pick a year earlier, with the Portland Winterhawks, for example. Still, considering 2012 and 2014, it looks like draft year quality of competition isn’t all that closely linked to pre-draft prospect quality: note that 2012 saw five CHL defensemen get drafted with the first eight picks despite the lowest quality of competition ranks, while this year, we’d be fortunate to see just two of the crop of shutdown-pairing blueliners in the top 10. Scouts aren’t perfect, but that’s a huge difference.

On the topic of Jones, he was advanced enough defensively to receive major minutes from new Caps coach Barry Trotz in Nashville in 2013-14, his draft+1 season. Jones, ranked fourth on his team in his final junior season, was able to make a successful jump to the NHL full-time, and Morgan Rielly (fifth) and Connor Carrick (sixth) made it to the NHL, too, while players like Scott Harrington (ranked 2-1-1 in his final three junior seasons), Stuart Percy (2-3-1), and David Musil (4-1-2) – admittedly, all lesser talents offensively – spent their draft+3 years in the AHL.

There are plenty of other cases where shutdown defensemen haven’t been able to make it in the NHL. Here’s a quick look at the first three rounds of the 2010 draft:

*+/- rel is relative plus-minus: the difference between a team’s plus-minus per 60 minutes with a specific player on the ice and with the player on the bench. You may know it better as Behind the Net’s “RATING.” It’s a rough measure of on-ice effectiveness, although it needs a large sample size to be accurate. Positive numbers are better.

Maybe here, Dalton Prout’s high quality of competition in 2010-11 and good results were indicators that he was better suited to handle the rigors of the pro game, but then again, Jerome Gauthier-Leduc had something similar going for him with Rouyn-Noranda and doesn’t look like he’ll amount to much professionally. On the whole, it does seem like more top prospect defensemen face tend to face top opponents, but it doesn’t really seem like it’s that big a deal — plenty of tough-minutes lower-tier prospects progress about as well as we’d have thought given their draft standing.

Putting it another way:

You can see my ratings here.

Although the numbers look pretty even, perhaps it’s worth noting that of the nine first-rounders who weren’t among the top-three in quality of competition and still project as significant NHL contributors, seven were highly regarded for their offense (the exceptions being Griffin Reinhart and Nikita Zadorov), while all four first-rounders who don’t look like they’ll be significant contributors soon are all (very much) defense-first. On top of that, the one later-round pick who’s looking pretty good, Madison Bowey, apparently developed quite the offensive game in his draft+1 season. Bottom line: if you don’t have a significant offensive game and don’t face top competition, especially in your draft+1 or draft+2 seasons, there could be a big problem.

On the other hand, it’s not surprising that the first-rounders are the ones still tracking to be good NHL players. There appear to be too many cases where those first-rounders are too gifted offensively to be given shutdown roles in their draft seasons, or are playing behind good 19- and 20-year-old prospects on their teams’ depth charts, to say that a high quality of competition ranking should be a significant draft ranking-changer; and having a middling or poor quality of competition is not a definitive indicator of a lack of upside, but it should be a flag for reconsideration. (Then again, if your team selects a project big crease-clearing one-dimensional defensive defenseman with its first-round pick, you probably knew at the time that it was reach, anyway.)

Context is always helpful, and it’s certainly not a bad thing if the 18-year-old you’re targeting is already facing top opposition. There likely is significant value to be gleaned from more detailed stats or different approaches — the Sharks have found enough to bother counting Corsi for major junior players, for example — but there’s too much noise or not enough significance in goals-based approaches for what we have so far to make a huge difference.

So here’s the payoff for those of you that have read this far (or just skipped ahead). The top defensemen coming out of major junior this season come in clusters, and we can try to differentiate some of the names, and based on quality of competition and offense…

Based on the approach laid out above, I’d be inclined to bump up DeAngelo, Honka — whom we selected with the 13th overall pick in the SB Nation mock draft on Tuesday — and McKeown, who all faced top opposition, had decent-to-sparking results, and showed us plenty of offense, too. If they hold onto the 13th pick and they choose to use it on a defenseman, the Caps should have a few good ones to choose from. And even if they trade back, they’ll likely have decent options.