Spaghetti and Meatballs, Not Apples and Oranges: Why Fancy Statistics and Eye-Tests Should be Considered Harmonious

By now, you've probably noted that there is a division between many in the hockey world about the use of advanced statistics. Part of this may be fear of the unknown or fear of moving away from the "old boys' club". Whatever the issue, we feel that the dichotomy between the analytical sides is unnecessary in the hockey world. Many domains, such as psychology, medicine, economics, or kinesiology, rely on both quantitative and qualitative data to help solve problems and advance their field. Hockey should not be the exception. Each method of analysis has its merits, which we will outline and show how they can be used to complement each other.

Merits of Qualitative Methods

What do we mean by qualitative methods for analyzing hockey? Put simply, if you're analyzing hockey qualitatively, you're critiquing what's happening by watching the game. This is probably how most of you have developed your critical eye for hockey. Essentially, whether you're conscious of it or not, when you watch hockey you're making judgments about players, strategies, refereeing, and the overall game. This is often referred to as using the "eye-test" or visually evaluating what's going on in a game or with your team. It involves observing what's happening and making a judgment based on your level of hockey knowledge and experience. The eye-test has been (and still is, for now) the main tool used for analyzing players and the game because it provides easily accessible data that most people in hockey can understand.

The eye-test helps when people want to evaluate a player's physical skills, such as skating, passing, or shooting. You need to be able to see a player's skills and how they use the tools that they've been given in order to determine whether they're at least capable enough to play in the NHL or on your team depending on your needs. Being able to visually scrutinize a player's physical skills is especially important when scouting because you're helping to decide if and when a player should be drafted or signed. If you're unable to see the nuances between a good skater and a mediocre skater, for example, you're going to have a difficult time scouting. The same can be true for evaluating their forecheck, backcheck, playmaking ability, defensive prowess, etc.

Coaches and management also require the use of the eye-test to determine their players' strengths and weaknesses. Indeed, coaches need to be able to physically see what a player is doing or not doing in order to adapt the player's game and fix (or hide) their shortcomings. Paul Maclean is probably sitting in his living room in Antigonish watching videos of the team from last year. He's watching how Jared Cowen could better position himself so he doesn't lose his opponent, how Colin Greening dishes the puck away too quickly, or how the Senators had problems breaking out of their own end. Through visual evaluation, he can then make lists of things that need to be worked on and come up with a plan on how to improve upon problem areas for the upcoming season. The eye-test is also important for coaches when they're putting together their strategies or changing strategies mid-game. Undoubtedly, coaches rely on the visual evaluation when reviewing videos to determine what needs to be changed on their own team and what other teams are doing; thus, through visual evaluation, they've created their game plan.

Merits of Quantitative Methods

Quantitative methods include anything that involves numbers: whether they're our good ol' fashioned goals, hits, and points, or the newer "advanced" metrics such as Corsi, Fenwick, and PDO. All of these numbers help quantify the events that happen over the course of the hockey game, such as assigning points for a goal scored, or assigning a +1 Corsi number for each shot attempt that happens when a player is on the ice. The merits of something like this is to have an objective counterpart to our eyes in order to confirm our belief, or to help us identify something that we didn't notice while we were watching the game. For example, if I was watching a game and thought "hmm, Clarke MacArthur looked good today", I could look at the data to see if I could find support for my qualitative observation. Well, it turns out that although MacArthur scored a goal in tonight's game - a memorable moment that stands out - his line was often hemmed into defensive end, as shown by a negative Corsi number, and he personally failed getting the puck out the zone, which can be shown in metrics such as zone exits. Quantitative metrics also help us understand why this happened. So, if we're continuing with our MacArthur example, was he hemmed into the zone because he played against tough competition? Well a quick look at some of the Quality of Competition metrics made publicly available, in combination with a glance at the possession stats for his opponents, would tell us that in fact, MacArthur's line was up against the line of Anze Kopitar that night, which routinely outshoots its opposition.

Now, this is a single game sample, and although quantitative metrics help here, their main benefit comes from looking at trends over a series of games. Using the infamous Toronto Maple Leafs example - metrics grouped together as hockey analytics such as the aforementioned Corsi, Fenwick, and a stat called PDO which looks at on-ice percentages predicted the Leafs eventual collapse in a way that the team's own coaching staff couldn't identify. How could established hockey minds not identify and solve the possession problem facing the Leafs? One potential reason is something that we all face as humans, and is based around how our brain is fundamentally designed for rapid processing of information, and this could lead to laziness developing in our line of thought that is often referred to as cognitive biases. Steve Burtch over at PPP discussed this at length here, and I strongly recommend checking his piece out, as it outlines a major reason why quantitative analysis is useful - it helps us combat these automatic biases by having an objective and rigorous method where we check and correct ourselves. The constant recording of every event lines up nicely with the scientific method, and allows us to go back and check where we went wrong if something doesn't add up with the number's we've recorded - something that isn't possible using just our memory for visual analysis, because it's prone to these biases, or second-hand information.

Why Both Should be Used

Let's finish off our Clarke MacArthur example to show how qualitative and quantitative methods could be combined and used by the coaching staff in order to help the player and make more informed decisions. So just to recap, Clarke MacArthur scored a goal - which made us think that he had a decent game - but after looking at the numbers, we realized that was all MacArthur and his line did, as they had negative possession numbers and a couple of failed zone exits. Further examination led us to examine MacArthur's quality of competition, where we saw that he matched up against one of the best lines in the league, Anze Kopitar's, and consider the Senators lucky that the goaltenders put up a fantastic save % in order to not get scored on while this matchup was in effect. Now we can ask the question: what went wrong in this matchup? Was there a gap in talent? Was there something MacArthur et al. were doing wrong that led to their poor possession numbers? A quick look at his season Corsi Rel QoC (a quality of competition metric) coupled with MacArthur's season possession numbers tell us that he usually does well against the best lines of the opposing team, so it's probably not a lack of talent. Oh, that's right, we remember recording that MacArthur and his linemates had a couple of failed zone exits against the Kopitar line, which led to more shot attempts against.

After identifying this through data, the coaching staff can key in on these specific plays through video and figure out what went wrong. This information can then be showed to MacArthur and his linemates through video, with instruction on what to do next time (short passes to players with support instead of shooting the puck up the boards!) in order to communicate their needs better to the player, instead of telling them "hey guys, you have a -10 Corsi tonight, smarten up", which doesn't do much for the player. This combination of quantitative (using data to identify the problem; negative possession numbers, high quality of competition, poor zone exits after tracking during the game) and qualitative (finding the specific plays on video) methods can help improve the team and limit possible biases that may have appeared using only one of these methods individually.

When critically analyzing the sport, both quantitative and qualitative evaluation are useful for providing support for arguments. Evidently, not all analysis needs both, it depends on the topic; however, some topics do require the use of both. The topic of possession-style hockey is a good example. You need to be able to see what it means to be a good possession team in order to truly understand it and explain it to someone, especially if you're a coach. Statistics can then be used to help bolster your argument by allowing you to show, with more accuracy than a mere eye-test, the offensive or defensive benefits of holding onto the puck.

Becoming the best hockey team has become akin to putting together a puzzle: you need all the correct pieces and they all have their correct place. Without being able to visually examine the sport of hockey, it becomes difficult to see what a team's needs are. That said, without using advanced statistics, your team runs the risk of not identifying everything that may transpiring on the ice. Ideally, both methods should be used as ways to advance the game. Both reveal things that the other doesn't, which means that both provide hockey organizations and analytical teams with different kinds of information. Neither encompass the totality of the game and perhaps nothing can because in all actuality, hockey is a game and games are unpredictable to a certain extent. What both methods do, however, is create a fairly inclusive composite for a team to use in order to lower the unpredictability of the game and provide themselves with a better probability to win.

Thanks for reading, enjoy your weekend!