Long Read: When Relief is Spelled G-O-A-L-I-E

We, and by “we” I mean the Dallas Stars media, the fans, and of course, us here over at the Bearded Ladies, have spent a gross amount of time bemoaning our backup goaltending situation, and not without good cause. There was over a year between starting wins for our back up goalies, which is unheard of. Was. Was unheard of, because the Dallas Stars did it. And now people have heard. Of that. Happening.

But there is another job for the backup goalie, and I feel like this one gets less fanfare: to come in and try to save the game when the starter is hemorrhaging goals.

I pulled the “In Relief” numbers because I wanted to prove that coaches used the goalie swap not only to replace a goalie on a bad night, but more frequently to send a signal to the rest of the team that their defense needed to step up and actually suppress some shots. I first noticed this trend while looking at Shots Against Per 60 Minutes for Enroth versus Lehtonen in one of the last games of the season. It was a day and night difference.

But of course, you can’t test a theory on just four games, so I pulled the last 4 years (2011-12 season until current) of “Goalie In Relief” game data for 8 teams. In the West, we have Dallas, LA, Chicago, and Minnesota, and in the East we have Columbus, Boston, New York (Rangers), and Montreal. This sample was chosen by my twitter followers without knowledge of the goal of the project, and therefore I consider it rather random. I removed all instances I could find of goalies being replaced mid-game due to injury, giving me 146 games worth of data points.

Separating Goaltending from Shot Suppression

How to test this theory, though? Ideally, the defense and goaltender are working in concert to prevent goals against. While GAA (Goals Against Average, or really, Goals Against Per 60) is frequently attributed as a goalie stat, Save Percentage has emerged as a preferred measure of goalie talent. And now war-on-ice also breaks down Save Percentage by shot type, leading to the use of High Danger Sv% as the best indicator of goaltending talent.

On the flip side, a goaltender has very little influence over the amount or quality of shots he faces, so we can assume that shots against are the sole responsibility of the defense.

Relief Sv% v Starter Sv%

This graph shows the weighted average Save % of each team’s starter (the orange line) compared to the relief goaltender. Remember, these are typically going to be a team’s worst games, so it shouldn’t be surprising that some of these numbers are so low.

Relief HDSv% v Starter HDSv%

This one looks pretty similar, some gaps between starters and reliefs widen, like Chicago, while others, like Dallas, narrow.

Relief SA60 v Starter SA60

And then this graph with weighted Shots Against Per 60 Minutes looks almost like the exact opposite of the previous two. Almost uniformly, the teams with large gaps in save percent, have small gaps in Shots Against, with one exception: Columbus. Dallas is also slightly odd, having a middling gap in both categories.


Finding a Pattern

But what does that even mean?

To better suss out what this kind of data means in relation to how a team performs, I took the delta (literally, Starter Stat – Relief Stat) of each category and put it in a chart to see what kind of patterns emerged.

Starter v Relief Delta Chart

Here you can see those Save % and SA60 gaps from the graphs a little better. The two lowest changes in Sv%, LA and NYR, have the highest changes in SA60, with the exception of Columbus, which fits my initial hypothesis. On the other hand, Chicago, Minnesota, Boston, and Montreal have high changes in Sv% and low change in SA60.

(We’ll get back to Dallas and Columbus in a second).

There are a few reasons that these gaps could exist. The first is always “it’s a coincidence.” The second, and more likely, is conscious coaching decisions based on players available and their motivations.

When looking at the Kings, Coach Darryl Sutter, who has been with LA for the entirety of this data sample, has admitted to using a goalie swap as a method of “waking up” his skaters. You can see from the TOI (time on ice) Delta, that he’s more inclined to pull his goalie earlier than later, which makes sense if that’s his goal. You can also see it works: LA has some of the biggest deltas (which is a good thing in this case) for Low and High SA60. During this timeframe, they posted the second best all situation SA60 in the league.

New York is slightly different. They have a Vezina winning goaltender in Lundqvist, who both Tortorella and Vigneault trusted to right the ship, pulling him only when necessary. If you remember from the first graphs, NYR has the highest “starter average” Sv%, another nod to Lundqvist’s abilities. Basically, if he can’t stop the goal, no one can, so the team focuses on shot suppression with the new goalie. While NYR only ranks 10th in SA60 for the last 4 years, they are ranked 2nd in Save %.

Probably the two of the more interesting looks are Chicago and Boston.

Both of these teams have the opposite problem, with Chicago leading in Sv% delta. You’ll notice that Quenneville (the only coach for the Hawks during this timeframe) is quick to pull his goalie when he feels he needs to, but Claude Julien gives his starter time to sort it out, much like the Rangers do. In both cases, the data supports this decision – Chicago is tied for 8th in Sv% over this time frame whereas Boston comes in at #1. Also, Chicago has a lot of goaltending depth – Scott Darling, Antti Raanta, Ray Emery and Corey Crawford have all posted perfect games in relief while playing 20+ minutes. Probably the most impressive of those was Darling’s 67 minute effort against Nashville in the first round of this year’s playoffs. On the other hand, Boston only has one goalie who has done that – Vezina winner Tuukka Rask.

That’s not what makes them curious, however; it’s the nearly complete lack of improvement in their defensive play. While Chicago does see some lift overall, it’s virtually nonexistent for Boston, and for both teams, Chicago especially, their Low Danger SA60 actually increase in front of the relief goaltender, leading me to believe that when these teams swap goalies, it’s going to be a very bad night for the Hawks and the Bruins.

Carolyn, what about Columbus and Dallas? They’re both either too high or too low to fit those patterns!

Well, yes. You’re right. They’re also the only teams on that list who have only made the playoffs once in the last 4 years – Dallas has admittedly had a lot of problems at both defense and goaltending, and Columbus is right there with us. Over the last 4 years, Dallas ranked 22nd in both SA60 and Save%. Columbus ranked 24th and 19th respectively.

There’s good news for Blue Jackets fans, though. Only 4 of 20 “goalie yanks” occurred this past season, with an average Starting Goalie Sv% of 79% and Average Relief Sv% of 92%, both higher than they have been, with only 0.75 Goals Per Game given up in relief, lower than the overall 0.95 average.

What Does All This Mean For The Stars?

There’s been a lot of reading so far, so let me repost the Delta Chart so you don’t have to keep scrolling.

Starter v Relief Delta Chart

I’ve been talking a lot about coaching for the other teams, so it stands to reason that the coach’s decision making is influencing Dallas just as much. Because this is four seasons worth of data, I put in two lines – Dallas’s total numbers, and Dallas’s numbers just under Lindy Ruff.

Under Ruffles, we can see a similar pattern as Minnesota start to emerge, with a larger Delta in Sv% and a smaller one in SA60. Remember how I suspected (along with most fans) that our lax defense was a large part of the problem on these “bad days”? Unfortunately, the data seems to suggest otherwise – that unlike LA, the issue can be pinned almost 100% on the goaltending. This is tough news to swallow, as goaltending is far more difficult to change, especially with Lehtonen’s $5.9m salary cap.

To break it out even further, here are the actual numbers.

Dallas Only Relief v Starter Stats

Under Lindy, you can see that the Starter’s Average Sv% drops from second highest in the sample, to just below sample average. Obviously, the large majority of these “yanks” were Kari Lehtonen, where he started 16 of the 21 games. This past season was especially brutal, with Kari being relieved 8 times. (I find this particularly funny because Lindy has spoken about how he doesn’t like to pull the goalie during a game, but nothing about his numbers with the Stars indicate this is true. He’s a practical man, our Ruffles).

On the other side of things, Dallas has had a lower SA60 under Ruff for both the Starter and the Relief, and he’s had far better success from the relief goaltender than his predecessor, Glen Gulutzan. This success is in large part due to Jhonas Enroth, who was perfect in relief for his entire stint with the Stars.

Remember how I mentioned Chicago had several goalies with perfect games in relief for 20+ minutes? Jhonas Enroth is one of two goaltenders in this sample who did that 3 times, and he did it all in one season. Also, the one time he was Relief for the Sabres this season he also acquitted himself well, posting a 93% Sv% while facing 38 SA60.

Does this build a stronger case for resigning Enroth? Well, maybe. In addition to breaking our starting back up curse, Enroth managed to sneak an extra win for the Stars out of one of his games in relief, winning over Florida in a dramatic shootout on March 5th. Still, Lindback chalked up two wins in relief, so I’m not putting much stock in that. Historically, Enroth hasn’t been fantastic for the Sabres, putting up a weighted average Sv% of 88% in relief, but he also started playing there at 23. And it was the Sabres. In all honestly, I think Nill will be shopping for a new back up come July 1st.

So where does all this data leave us?

Hopefully with a better understanding of how our coach approaches the “bad days”, and also with a sense that the bad days are getting better. Defensively, the Stars are allowing less Shots Against overall under Coach Ruff. This season especially, they had moved from 25th in SA60 for the first half of the year to 13th for the second half.

The lingering question is going to be Kari – if he can get back to form, hopefully we’ll start seeing more normal numbers of “relief games.” The average for this sample was between 4 and 5 games a year. 8 in one season is very high.

Cross your fingers Stars fans, and let’s hope Jim “The One” Nill chooses wisely this offseason.

Bonus Chart and Data Caveats

Where possible, I used averages weighted by time on ice, so that games where reliefs/starters played more had a greater impact on the respective stat. Unfortunately, because of the way the Low, Medium, and High Danger Sv% are calculated, I was unable to weight those averages, and instead used a straight average.

Because these averages are weighted, things like TOI won’t always add up to what you would expect a total to be (eg 60 minutes).

Because a certain number of shots are “unclassified”, adding Low, Medium, and High Danger shots will not automatically add up to Shots Against, nor will their respective rates add up to SA60.

Columbus had one game where a relief goaltender faced 0 shots against, which greatly affected their numbers, bumping up their relief weighted Sv% from 89.5% to 92%, but because of the large sample size, had little effect on the overall averages.

OK! That’s that! Now for the BIG chart for the real dataheads!

Relief v Starter Stat Chart


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s