As you may have heard mention on twitter or on our podcast, the Sloan Sports Analytics Conference took place a few weekends ago. By all accounts, it was mostly analysts preaching to the choir of fancy stats converts, but like most events, they decided to inject a little drama into the proceedings by having their last panel include a dissenting voice: Brian Burke, President of Hockey Operations of the Calgary Flames.
At this very panel he was quoted as saying “Numbers are overrated a lot of the time … It’s an eyeball business … You’ve still got to watch guys … No one’s ever won a title with Moneyball.”
…Right. Analytics definitely didn’t help both the Kings and the Blackhawks win multiple cups.
Though the tweet seems to have been deleted* Patrick Burke went on twitter after his father’s comments saying something to the effect of “Burkie uses analytics, he just calls them statistics.”
Listen, we all know there will be old men clinging to the ways of the past no matter what industry you’re in. Those people are becoming few and far between. But what is concerning to me is the amount of people who think that analytics are the same thing as statistics.
Let me make this clear right now:
Statistics = numbers
Analytics = the interpretation of numbers, designed to improve the understanding of behavior and the world around us, and to use this understanding to predict future outcomes
Too frequently, beat writers and bloggers will post statistics without context and expect the numbers to speak for themselves. This is never the case, even when it appears self evident, for two reasons. The first is that without context, the number may be deceptive.
Here is a recent example from the game against Tampa Bay on Saturday.
Erik Erlendsson is one of the Tampa Bay beat writers. At first glance, it appears that Dallas should’ve been a lock to win that game. 21 wins when they lead after 2 periods! And Tampa has only come back from losing once! For Tampa Bay fans, these stats would make sense, as the Bolts are generally perceived as a team who are great at building a lead, but bad at overcoming a deficit.
However, Stars fans might look at that number and go “hold up.” I know I did.
When you pull up this chart and sort by wins, Dallas is mid pack, in the company of some excellent teams like Detroit and Chicago. But we, the Stars fans, know that we are not as good in the 3rd period as Chicago (who has never lost after leading at the end of the 2nd).
And this is where context becomes important. We may have 21 wins if we lead after the 2nd period. But if we sort this chart by Winning % (mathematically, # of Wins / Number of Times Leading), it tells a very different story.
Ahh, there it is. This is where Erlendsson’s tweet falls apart. Dallas has 21 wins because they frequently go into the third with a lead. But per this chart, they’re actually one of the worst teams at winning games they’re leading. So is it really an #UphillClimb for Tampa when they’re facing the 3rd worst 3rd period team in the NHL?
And, if we’re going to be objective, Tampa’s situation isn’t even as bad as it seems upon first look.
If you sort this chart by losses, this view lets you know how many times a team finds themselves in a situation where they’re down after two periods when trailing after the 2nd period. You can see a pretty clear delineation of teams at the bottom of the league, mid pack, and top of the league. Tampa’s 18 losses are actually pretty average. And if you’re only looking at wins, well, there are some good teams with even worse records, like Pittsburgh, Winnipeg, and Washington.
The second problem with analytics is that without context, you’re expecting the general public to draw the same conclusion from a chart or a statistic that an analyst would.
As much as I love the hockey analytics community on twitter and find their work fascinating, most of them need a non-analytics driven editor, like I have (love you, Merrin). The average hockey fan doesn’t spend nearly as much time as us nerds staring at numbers, so when Montreal didn’t even show up against the Kings last weekend, the analysts sat around scoffing, and a lot of fans were bewildered at their reactions.
You see, Montreal has a Score Adjusted CF% this season of 49.3%, sitting right between Philly (50%) and Arizona (49%). It is very difficult for a team to score when they’re not possessing the puck.** Largely, Montreal’s wins have be coming on the back of the excellent goaltending of Carey Price. And if he has a bad game, they’re not going to win. Further evidence of this is Montreal’s league leading PDO of 101.7. And as you probably know, PDO = Shooting % + Save %, and is generally considered the measure of Luck. To be this ‘lucky” going into the playoffs is unsustainable. Just ask the Avalanche.
Often you’ll find self-labeled analysts taking short cuts in their articles, like not defining common terms (like PDO or Corsi/SAT), or not labeling axes on charts. Again, if the intended audience is not other analysts, these are mistakes that can be extremely confusing and perpetuate the belief that analytics are a bunch of people just making shit up to look smart.
Obviously, I see a ton of value analytics, and most of the hockey world agrees with me. But analytics is more than just numbers, and unless conclusions are presented logically, concisely, and with context, all you’re getting are statistics. And that’s not the same thing.
*I am fully willing to believe this is not intentional, as all of Patrick Burke’s tweets from February and January are missing, which is just weird.
**Unless it’s the Stars.