The Quality of Theodore's Consistency
by David Getz on Apr 7, 2009 12:20 PM EDT
Email
Print
Comparisons across sports may not always fair, but they are inevitable. One of the more intriguing such cross-sport parallels is that between pitchers and goaltenders. It's certainly one that make a certain measure of sense: both positions entail duties that differ drastically from those of teammates, pitchers and goaltenders are the ones who get wins and losses tacked to their record, they're the ones who can most easily be labeled either "hero" or "goat", the ones whose performance can single-handedly win games on a regular basis, and the ones whose roles are often defined as "just be good enough to give you team a chance".
It's that final caveat that has give rise to baseball's Quality Start statistic as a way to measure a pitcher's ability to consistently give his team a good chance at winning a game. The Quality Start concept has now been adapted to hockey by Puck Prospectus, which defined a Quality Start for a goaltender in the following way:
Based on these two parameters, a Quality Start is defined as any start in which the goaltender's save percentage is .913 or better, or at least 0.885 percent, but allowed fewer than 3 goals. Looking at every game this season, Quality starts have resulted in an actual winning percentage of 0.775, while non-Quality Starts have a winning percentage of only 0.325.
The numbers are not arbitrary - they're based on league-wide winning percentages and median league performances, and the logic and methodology seems to be sound. Despite this, there's no doubt that the metric is imperfect - on Friday night Simeon Varlamov was not close a Quality Start by either measure, yet was not clearly to fault on any of the five goals he allowed - but it does do a better job of analytically measuring goaltender consistency than any other metric. So how does Jose Theodore stack up with goalies on other Eastern Conference playoff contenders? Here's a look at the season-long statistics through last weekend's games:
Simply put, these numbers don't flatter Theodore. Actually, they're all but disastrous and seem to be a legitimate cause for concern. Sure, we figured he'd be behind guys like Henrik Lundqvist and Tim Thomas, but Martin Biron and Carey Price? Whatever the measure of performance, you have to be concerned if they guy you're relying on is behind those two, right? Probably, but before we start throwing our hands up and inciting panic, there are three important facts to consider.
The first is that the 2008-09 season has truly been a tale of two season for Theodore, the clear cut divider being the December 23rd game against the Rangers in which Theodore was pulled in favor of Brent Johnson about twelve minutes in to the game, and then returned to the net in the second period. At the end of that first period, Theodore was sporting an unsightly 3.39 goals against average and a .878 save percentage for the season. Since returning to the net, Theodore's goals against average has been 2.55 and his save percentage has been .911. Unsurprisingly, Theodore's quality start numbers have risen as well. The December 23rd game was his 17th start of the season, and at that point Theodore has only registered five quality starts; his quality start percentage was an embarrassingly low 29.4% Since that game, Theodore has started 36 games and registered 19 quality starts, a quality start percentage of 52.8%. It's also worth noting that on one start, January 20th against Ottawa, Theodore missed a quality start by the slimmest of margins, recording a .912 save percentage.
The second fact is what quality starts attempts to measure, being good enough to give your team a chance to win, is something that has a very different definition from team to team. Backstopping a team that plays a defensive style and doesn't score all that often like New York (2.37 goals per game) or Florida (2.74 goals per game) is quite different from backstopping a team that favors a wide open, high scoring system like the Capitals, who average 3.20 goals per game, and it's a difference that this statistic doesn't capture. In the crudest terms, Jose Thedore's not going to have win very many 2-1 games for a Capitals team that scores three or more more often than not, while the numbers suggest that giving up twenty goals in a seven game series would be enough to eliminate a team like the Panther or the Rangers.
The third is that the table looks radically different if it's based on career playoff performance* rather than this season:
*Only playoff performances from the 1999-2000 season onward are included.
Of course this particular data set if rife with small sample sizes and data being skewed by hot streaks (would anyone really take Cam Ward or Marc-Andre Fleury over Martin Brodeur or Tim Thomas because they had one hot postseason?), and realistically Theodore is probably not the fourth best of these nine goalies. In addition, Theodore's numbers are skewed from his time Montreal. Post lockout , Theodore has started 19 games and had nine quality starts, a 47.4 quality start percentage. That said, his past playoff performance certainly suggests he will, at the very least, be competent Capitals this postseason, especially considering that four of his post-lockout non-quality starts came last year against the Red Wings, when Theodore was sick enough that he was once sent back to the team's hotel before the game ended.
So what does this all mean for the Capitals? Well, for starters it means that the Capitals probably aren't too far behind other Eastern Conference teams when it comes to how much faith they should have in their goaltending (which isn't exactly new information) and that the steady stream of criticism leveled against Theodore this season is largely unfair. But most importantly it means that while Theodore might not be all the likely to steal a ton of game or a series for the Capitals, he's even less likely to be the reason for an early postseason exit.
44 comments
|
0 recs |
Do you like this story?
Comments
Normally, statistics make me feel a little better and more secure about our position going into the playoffs. Not this time. I actually thought Theodore would stack up better against the competition post-Rangers game. But since then, he still has a lower QS% than Carey Price? Not encouraging, even given the different systems the teams run.
by Pivonka, Michael Ridley on Apr 7, 2009 12:56 PM EDT reply actions
Why?
It made me feel about the same, because it more or less reinforced what I already believed: that Theodore wasn’t going to carry the team to the promised land on his back, but that people who were afraid the team was going to lose because of Theodore’s play are being paranoid.
It shattered any notions I had that Theodore had improved his consistency, or that he stepped his game up in the playoffs.
The only shrivel of hope left is the “he wants to prove the world wrong!” angle. But he has been trying to do that for a few years now, and he still gives up soft goals, huge rebounds and gets caught out of position.
I guess the real hope is that the defense in front of him breaks the habit of giving up four odd-man rushes a game, and Tom Poti decides it’s no longer a good idea to send limp-wristers up the middle.
The keyboard is mightier.
Is any of this really shocking to anyone that follows the team closely? The Capitals key to success is consistently average goaltending with 1-2 big saves a game. Theo has shown he is the man for that job since the infamous NYR game.
This team has shown they can lock down and get the job done. BUT, they are at their best when they are forechecking hard all game while tiring the other team down in the offensive zone.
by zephyr on Apr 7, 2009 1:45 PM EDT reply actions 1 recs
Just win, baby!
1) Nice/interesting post. It merely confirms what we have read from many Caps fans; The jury is still out on JT60. Its all about winning in the post season!
2) If he can lead them to a conference title,(or better) few will care how mediocre his regular season numbers were. If they lose in the first round, the finger pointing/second guessing will be intense!
Hmmmm. Interesting metric. I don’t really like the QS as a baseball statistic, and my initial reaction is that QS for hockey is a little rudimentary. My main quibble is that the inquiry proceeds like this: 1) we know that when goalies attain either A or B results in two stats, they win 77% of the time; 2) did Goalie X attain either of those milestones? 3) if yes, QS; 4) if no, non-QS.
Well, yeeeesss…but HOW did he get there? Did he stand on his had for 3 periods and slam the door shut against a determined onslaught, or did he fend off 24 weak wristers from the half boards and let the 25th flutter past him?
We really need to know how the defense played, how the opposition played, the quality of shot against, the quality of save required, and then the quality of positioning, rebound control, and more.
Much like in baseball, defensive stats in hockey resist simple statistical analysis. However, there is progress being made in baseball in the development of meaningful defensive metrics, so I am hopeful for hockey in this sense.
As for Theo, we are left with that most untrustworthy of evaluation tools, the eyes that God gave us.
But this stat is a start in the right direction. It’s more flexible than just SP or GAA, and certainly better than W/L record. So for that I am pleased.
This is random, but I don’t know where else to put it:
I bought the NHL game of the year Penguins v. caps game from October 11 from iTunes. I paid my 1.99, and am now subjected to the horrible penguins announcers who for some reason thought Tomas Fleischmann’s name was Tomy. This makes me hate Pittsburgh even more, somehow.
The way quality-starts are determined for this comparison is as worthless as comparing two pitcher’s ERA from different teams.
THOMAS: 37 QS, 34 Wins. Team is -3 for Thomas’ quality starts
LUNDQVIST: 44 QS, 35 Wins. -9
FLORIDA GOALIES: 46 QS, 39 Wins. -7
BRODEUR: 17 QS, 17 Wins. E
WARD: 41QS, 38 Wins. -3
PRICE: 25 QS, 23 Wins. -2
MAF: 29 QS, 33 Wins +4
BIRON: 24 QS, 27 Wins +3
THEODORE: 24 QS, 31 Wins +7
We can debate all day long about what a quality start is. In just calling it a quality start leads one to believe that it is relative to the team they are on, so to me using league-average stats to determine a player’s quality start is kinda silly. I would much rather see something where QS is based on a player’s GAA PER GAME vs a team’s AVERAGE GOALS FOR on the year. Afterall, would the Cap’s view a quality start the same as the Blue Jackets?
But comparing these two methods of QS computing raises a unique question. In trying to rank players based on either stat, which to you use as the “high water mark…?” Is it better for a goalie to have more quality starts and less wins, or more wins and less quality starts?
Did the team steal a win from a poor quality start, or did the team fail to capitalize on a good quality start?
Using the one I suggest, JT’s got 37 quality starts and 31 wins, meaning the team failed him 6 times.
If I had a lot of money/time, I’d develop a team of folks dedicated to supplying the sports world with better stats than just looking at a box score and putting shit in a spreadsheet. Example. Varly’s start vs the Buffa-slugs. 2 goals they scored were actually “real” goals. The other 3 were flukes of nature that deflected 1-3 times before reaching the twine. Should that kind of stuff count against a goal tender? Simple goal-by-goal analysis of a game could yield much better data for goalie comparison. How much lower would JT’s GAA/SV% be if Poti/Juice/Schultz’ fat asses weren’t in his face half the time?
by FFSEnough on Apr 7, 2009 2:55 PM EDT reply actions 1 recs
Expanding a little
Staring at the numbers I posted above… Team +/- vs QS…
It almost jumps out at you that a negative +/- here means the goalie is better than the team as a whole. Teams with a positive +/- here indicate that the team as a whole is better than the goalie. Would anyone argue that for each element of the list? (Brodeur skewed due to small #gp)
Rec’d because I like showing actual wins next to quality starts. I agree that you can’t really compare goalies without somehow adjusting for team performance. Because the Caps score a lot, for example, maybe we should calculate a QS for Theodore by including games when he allowed 3 goals but saved at least .885 percent of shots (I think the metric above counts only games where less than 3 were allowed). If I had the time, I’d try to calculate it, but my gut tells me his QS percentage would be a lot higher, and more reflective of whether he actually have the Caps a chance to win (which is the goal of this stat, right?).
by Scott in Shaw on Apr 7, 2009 4:03 PM EDT up reply actions
It probably would but is it fair to reward/punish a goalie because his team can score more? Save percentage and goals against are the only real individual stats you can compare among goalies so if you are trying to identify quality goaltending it just makes sense. Sure, there is team impact involved in those numbers but not as much as judging a goalie’s quality starts by how many goals his team scores.
QS is a stat based off of Save% and GAA, yet attempts to normalize it to some league-wide average of what gives the team a good chance to win. Does it really make sense that GAA/SV% is such a better measure of a goalie’s individual performance, yet a stat that is based on GAA/SV% makes much less?
GAA and SV% are curious stats that include ALL of the negative aspects of a team’s play infront of the goalie, but include NONE of the positive parts of the team’s play. That’s quite an interesting conundrum to base one’s feelings about a goalie’s performance on. By normalizing quality starts to include a team’s offensive capabilities, you get closer to creating a metric that includes the team’s total contribution to their goaltender’s effort. Something GAA/SV% falls woefully short on.
There are a few stats in hockey that attempt to indicate team performance, but GAA/SV% feel waaay too much like baseball pitching metrics that will never apply to a true team sport.
by FFSEnough on Apr 7, 2009 10:57 PM EDT up reply actions 1 recs
If you are trying to evaluate the goalie then I suggest you should be ignoring the positive parts of the team play. If you are trying to roll all the team variables you can into one metric then all you are looking at is Ws.
If you ignore the positive, you must also ignore the negative, or the stat is fundamentally skewed by the quality of the goalie’s teammates and the style of hockey the team plays. I’ll use CLB as a shining example and try to anologize to the NFL.
Ken Hitchcock is one of the most successful coaches at “shut-down” hockey. His teams are constantly able to keep the opponent’s offense tied up. Marty Turco was one of the best goalies GAA/SV% when Hitch was there and the team was very successful. In CLB now, we are starting to see what a decent goalie and Hitchcock’s style can do for a player’s GAA/SV%. Part of the reason I’d prefer to see Rinne get the Calder, since his team’s style is much less suited to padding a goalie’s numbers than CLB. The question I have in my mind is, if you put Mason on a less defensive minded team, would his gaa/sv% be as good? Is it ALL him or does Hitch’s system have something to do with it.
Last year in the NFL, we got a good look at the “Is it Brady, or Bellichick?” question. The answer to that question will be proven this year in KC.
To truely evaluate a Goalie’s performance and be able to compare it across teams (GAA/SV% can be used to compare two goalies on the same team provided you can normalize for competition) you either need to include all aspects of the team’s contribution to the goalie’s stats, or eliminate all of them. It’s far easier to include them all (Quality starts based on team’s G/G) than eliminate, as elimination of the team’s contribution would require analysis of every goal/shot for quality and “what should have happened.” The “what should have happened” goes far beyond just a mere “he should have put that rebound in the corner…” You have to look at where the shot was taken from and what the goalie could have possibly done with it.
Again, if I had the time/money, I’d attempt to take up that endeavor. Included with the GAA/SV% normalization I would include a “goalie saved their asses” vs “goalie let the team down…” Saved their asses would include Goalie beats odd-man rush + save on a screen/deflection shot + positive cross-crease results. let the team down would be unscreened shots beating the goalie/bonehead plays/actual poor rebound control/crease coverage normalized for shot quality. There’s so much more that could be done to generate individual comparison stats for this game, but it requires time and a lot of thought to polish them so they answer the questions you’re trying to ask.
Just Win Baby
I have to admit that list made me a little queasy, but then I realized ( unless I missed it) that it doesn’t seem to take actual wins into account. So here’s how it looks when you reorder the list based on regular season wins:
Cam Ward-38
Henrik Lunqvist-35
Tim Thomas-34
Marc-Andre Fleury-33
Jose Theodore-31
Marty Biron-27
Tomas Vokoun-24
Carey price-23
Craig Anderson-15
Martin Brodeur-N/A
Theodore looks a little better there, but what really counts is playoff wins. When you order it that way aside from Brodeur, who has a ridiculous 95, Jose is right at the top of the list:
Jose Theodore-19
Cam Ward-15
Marc-Andre Fleury-15
Henrik Lunqvist-11
Marty Biron-9
Carey price-5
Tomas Vokoun-5
Tim Thomas-3
Craig Anderson-0
Now admittedly, Jose has a below .500 career playoff record but so do most of the other guys on the list. The bottom line is that in the playoffs they never ask how you looked, they only ask did you win? And in that regard, Jose is looking pretty good.
I tend to agree that “quality starts” tends to both oversimplify and overthink things. A quality start for the Caps and for the Wild are two different animals entirely, and it’s hard to treat them alike in my book.
Bottom line: did you give your team a chance to win? Yes? Then it was a quality start.
Japers' Rink: Hockey blogging from the most powerful city in the world
by J.P. on Apr 7, 2009 3:50 PM EDT up reply actions 1 recs
did you give your team a chance to win? Yes? Then it was a quality start.
Exactly. When it comes down to it that’s what being a goalie is all about.
I always come out and play with that type of mentality.
This is an interesting exercise, but I don’t think it tells us much that couldn’t be gleaned from looking at a team’s/goalie’s numbers and their records. The quality of the defense and type of system that the team plays simply skews things way too much. The post-Miracle on 34th numbers for Theo are about what I’d expect.
My eyes tell me that there are somewhere between 4 and 7 goalies in the East that I would say are better than Theo, but only 3-4 (Thomas, Brodeur, Lundqvist, Ward the way he is playing right now) that are significantly better in my estimation. Theo is still good enough to backstop the team to some series wins and get hot in the playoffs. If the Caps win one or more series, it will be because the rest of their skaters are better than the other team’s. That’s how this club wins and was built to win, and the playoffs won’t be an exception.
If the Caps win one or more series, it will be because the rest of their skaters are better than the other team’s.
Isn’t that how it almost always is? How often does the best goalie in the playoffs win the Cup? It happens, and a hot goalie definitely helps a team, but Roy is known as one of the best big game goalies of all time and he has 4 Cups in 17 playoffs; Brodeur has 3 Cups in 13 playoffs. As good as they are/were they don’t win singlehandedly. When was the last time you can remember a team winning the cup solely because of their goalie? I can’t remember a team that wasn’t competitive 5 on 5 winning just because of the goalie. Hasek in Buffalo is the closest I can come and they never won the Cup.
Sure, I guess this is true. But, to be clear, I’m talking just in terms of winning a few rounds, not necessarily the Cup. I don’t have enough time to look into this myself but I think many (particularly defensive-minded) teams that have won 2 rounds in the playoffs have had pretty statistically great goaltending for the rounds they’ve won. I’m not sure I can name that many teams that won the cup because of their goalie, but I can think of several offhand who went deep in the playoffs in large part because of spectacular goaltending (Ward when Carolina won, Giguere when the Ducks won/lost on the finals, Brodeur pretty much every time the Devils went deep, Hasek with Buffalo, etc). Point is, teams getting outstanding goaltending can sometimes beat a team whose skaters are a little better, at least on paper. While I think Theo will do just fine in the playoffs, I don’t think the Caps are going to win any series because Theo is “standing on his head.” Which isn’t to say they can’t win them – thankfully the Caps have arguably the best roster of skaters in the East. I’d love to be wrong and see Theo dominate, though.
If there’s one thing I’ve learned over the years, it’s that analysis of any kind is usually worthless going into the playoffs.
I emphasize going into, because I do think performance from one series to the next is “trendable”, but there really is a different season when the playoffs begin. What teams (and players) did or did not do in the regular season makes little difference in the post-season.
Let’s all hope that means that Theo will make all of us look silly that we were even having this discussion. :-)
I’m not sure I buy the quality start statistic. According to this metric, stopping 16 of 18 shots is more impressive than stopping 30 of 33, etc., and I’m just not sure this is true. This is the kind of statistic that makes a Caps goalie look worse than their raw statistics might, simply due to Boudreau’s system and the odd man rushes against that (unfortunately) come with it.
I wish a “big save” statistic existed – perhaps this might increase my confidence in JT.
All I can tell you is that JT is the starter on the second best team in the east and Huet’s the backup on the 4th best team in the west.
I gotta say I was surprised when I got to the last paragraph and saw the conclusion that this was supposed to be reassuring for Caps fans. I have to think what’s been happening this year, what’s been happening in his time as a Capital is way more important than what he did years ago behind completely different defensemen for completely different teams.
(And I’m surprised Biron’s playoff numbers are so bad, considering he’s won 2 of his 3 career series).
Swing by The Flyer Frequent. You have nothing better to do.
I’m not surprised about Biron. He’s been much more lucky than good. Whenever you see someone who’s late-season and playoff records appear that much better than their regular season records, that guy is due for a regression to the mean.
by Gould Old Days on Apr 7, 2009 5:13 PM EDT up reply actions
Actually the main surprise for me is how good Carey Price’s numbers are. He played some absolutely shitteous hockey last playoffs. I guess he snuck some good games out vs. Boston, but he just pooped out some really awful starts.
Swing by The Flyer Frequent. You have nothing better to do.
by Ben Rothenberg on Apr 7, 2009 5:25 PM EDT up reply actions
I’m surprised Biron’s playoff numbers are so bad, considering he’s won 2 of his 3 career series
I think that just goes to show how ummm young this stat is and that in it’s current stage, shouldn’t be taken by itself. There are too many contingencies…as stated above highlight saves, soft goals…quality shots vs. shots that are routine…goals that are scored during a 5-on-3 (almost never the goalies fault IMO).
However, I think the stat is a brilliant starting point…but one that needs some work.
The PPGs are a good point. Didn’t think of that.
Swing by The Flyer Frequent. You have nothing better to do.
by Ben Rothenberg on Apr 7, 2009 5:22 PM EDT up reply actions
Comments For This Post Are Closed
Secondary Sidebar
User Tools
Related FanPosts on Japers' Rink
FanPosts
Community blog posts and discussion.
Recent FanPosts
FanShots
Quick hits of video, photos, quotes, chats, links and lists that you find around the web.
Recent FanShots
Southeast Standings
| GP | W | L | OTL | PT | |
|---|---|---|---|---|---|
Florida
|
53 | 25 | 17 | 11 | 61 |
|
Washington
|
54 | 28 | 21 | 5 | 61 |
|
Winnipeg
|
56 | 26 | 24 | 6 | 58 |
|
Tampa Bay
|
53 | 23 | 24 | 6 | 52 |
|
Carolina
|
55 | 20 | 25 | 10 | 50 |
(updated 2.10.2012 at 9:27 AM EST)
The Vault
In Association With
Latest Puck Daddy - NHL from Yahoo! Sports
- Puck Previews: Blackhawks reeling; Sabres streaking; Holmstrom’s 1000th game
- Will Teemu Selanne remain with the Anaheim Ducks through the trade deadline?
- Nikolai Zherdev takes stick to crotch, elbow drop in KHL game (Video)
- Puck Headlines: Accused coach Pravilov dead; more Tim Thomas heat; Junior team goes nuts post-shootout
- After Scott Gomez scored, other candidates for NHL goal drought obsessions
Latest stories and headlines from NHL.com
- Blue Jackets place Lebda, Methot on injured reserve, add Prout, York on emergency recalls
- Blue Jackets put Lebda, Methot on injured reserve
- Lightning's Dominic Moore fined $2,500
- Detroit goalie Howard sidelined by broken index finger, hopes to practise soon
- Howard misses 4th game because of broken finger
Certain photos copyright © 2012 by Associated Press or Getty Images. Any commercial use or distribution without the express written consent of Associated Press and Getty Images is strictly prohibited.
