The Sabermetric Revolution: Assessing the Growth of Analytics in Baseball (16 page)

For instance, the NBA’s New York Knickerbockers hired Dave Heeren in 1961 as the team statistician. In 1958, Heeren had developed what he called the TENDEX formula for evaluating players.
⁴
In its initial version, a player’s TENDEX rating equaled the sum of his points, rebounds, and assists, minus missed field goal attempts and missed free throw attempts. Heeren evolved the formula into more sophisticated versions over the years. TENDEX became the basis for the NBA Efficiency rating and for subsequent linear weights metrics of player and team performance. It is unclear, however, to what extent the Knicks or any other team made effective use of this rating system, and, as we shall see, the various incarnations of linear weights in basketball all have significant limitations.

Heeren did not begin to publish articles on his system until the late 1980s. He published his first TENDEX book in 1989. Dean Oliver, Bob Bellotti, and John Hollinger began writing in the 1990s, primarily on websites, suggesting various refinements and elaborations to TENDEX, as well as other evaluation methodologies. The proliferation of articles, books, and websites using quantitative analysis of basketball, however, did not begin until after 2000.

Also after 2000, NBA front offices began to embrace the notion that they could improve team performance by a more sophisticated application of statistical analysis. The practice of basketball analytics has evolved cautiously since then. Soon after he purchased the Dallas Mavericks in 2000, Mark Cuban hired decision sciences professor Wayne Winston, his former teacher at Indiana University, as a stats consultant. Cuban seems to have been first basketball owner to explore the application of analytics. Cuban’s explanation for his innovative move is straightforward: “I wanted to give the Mavs any advantage that I possibly could.”
⁵

Upon finalizing his purchase of the Celtics in January 2003, Wyc Grousbeck hired Daryl Morey as vice president of strategy and information to do statistical analysis. Grousbeck’s background was in biotech and software venture capital. He realized that the Celtics’ front office was in dire need of overhaul and modernization, and he wanted to exploit every opportunity to give his team a leg up on the competition. One of the first statistical insights to benefit the Celtics was the discovery that twenty-four of the previous twenty-five NBA champions had three all-star players on their roster,
⁶
and, most important, one of the three was a league MVP or Top Fifty all-time player. Eventually this insight led to the Celtics’ plan to get Kevin Garnett.

In October 2004, the Seattle SuperSonics hired Dean Oliver as a full-time quantitative analyst. In 2005, in preparation for the NBA draft, the Portland Trailblazers commissioned Protrade Sports (subsequently acquired by Yahoo) to develop a model to predict how college players would perform in the NBA based on drafted players over the previous ten years. In December 2005, Cleveland Cavaliers GM Danny Ferry hired Dan Rosenbaum as a statistical analyst. In April 2006, the Houston Rockets hired Daryl Morey as their assistant general manager and announced that Morey would become the general manager during the next season. During the 2009–10 season, Aaron
Barzilai, Alex Rucker, Ryan Parker, Jon Nichols, Kevin Pelton, and Ken Catanella, all from various analytic websites, were hired to do statistical analysis in the NBA. Interestingly, most of the hires were of people participating on the APBRmetrics bulletin board.
⁷
After the hires, the activity on the board diminished appreciably.
⁸

And so it went, step by step. During the 2010–2011 season, Benjamin Alamar, founder of the
Journal of Quantitative Analysis in Sports
and team statistician for the Oklahoma City Thunder, estimated that roughly half of the teams had a “metrician” working in their front office. And, prior to the beginning of the 2012–2013 season, Dean Oliver, now working as the director of ESPN’s analytics department, identified at least twenty-two NBA teams with a metrician. Further, Oliver anticipated that two more teams were about to add quantitative analysts.

In mid-December 2012, the new ownership group of the Memphis Grizzlies poached John Hollinger, best known for developing the comprehensive Player Efficiency Rating, from ESPN to become its vice president of basketball operations. In his farewell column for ESPN, Hollinger offered the following assessment:

It’s hard to believe this is true, but just eight years ago very few teams showed any interest in analytics, and those who did wouldn’t admit it publicly. Seriously. Teams employed analytics people they wouldn’t even mention in their directory for fear of ridicule.

In less than a decade, teams have reversed course: Now, if anything, many try to promote how much they’re doing with analytics. At least two-thirds of the league’s teams have invested in this area, and while a few of them are just checking a box, most are seriously committed to it.
⁹

In professional football, owing primarily to the more continuous and interdependent nature of play and very sparse data, the incorporation of analytics into team front offices has been more halting. Nonetheless, basic statistical analysis has been around for decades in the NFL. Bud Goode worked analyzing statistics for twenty-one different teams over the years and carried out
some rudimentary quantitative tests. He found, for instance, that yards per pass was the strongest predictor of team win percentage among a variety of game stats.

In 1971, an article co-authored by former Chicago Bears quarterback Virgil Carter in
Operations Research
, based on data from the first half of the 1969 season, plotted the net number of points scored when a team had a first down at different field positions. For instance, the article found that the net points on average when a series began on the team’s own five-yard line was minus 1.2 points (i.e., the team on defense was more likely to score than the team on offense) and 6 points when the series began on the opponent’s five-yard line. The relationship was pretty much linear between these two points. This article adumbrated a line of research that began in earnest with Bob Carroll and Pete Palmer’s 1988 book,
The Hidden Game of Football
, and continued with David Romer’s 2006 article that will be discussed below.
¹⁰

Evaluating individual performance in football, however, has always been tricky. An individual’s stats, whether completion percentage, yards per pass, yards per rush, interceptions, or tackles, depend heavily on the work of one’s teammates. The outcome of a play in football is essentially the result of a matchup of eleven players against eleven players, whereas in baseball it is basically a one-on-one matchup. It is perhaps for this reason that practitioners of football metrics have yet to appear widely in NFL teams’ organizational charts. The New Orleans Saints moved beyond the use of magnetic boards during the league-wide amateur draft to the use of a computer program that electronically depicted the draft’s progression. The San Francisco 49ers hired Paraag Marathe, a Stanford MBA and former Bain Capital employee, to do statistical analysis back in 2000. Today, he is the team’s COO and Brian Hampton does some work for the 49ers with analytics. The Philadelphia Eagles, New England Patriots, and Dallas Cowboys are also known to have put some emphasis on statistical analysis in making personnel and game strategy decisions. Further, prior to the 2012–2013 season the Jacksonville Jaguars and Baltimore Ravens (on the way to their Super Bowl victory) hired stat analysts.
¹¹
And the Buffalo Bills announced in early 2013 that they are “going to create and establish a very robust football analytics operation that [is] layer[ed] into our entire operation moving forward.”
¹²
But, beyond
these examples, there is little evidence of widespread employment of analytics staff, let alone of the integration of analytics into decision-making, as a way to achieve competitive advantage in the NFL.
¹³

The New England Patriots may be the exception that proves the rule. Although the details of the internal workings of the Patriots front office are proprietary, it appears as though the Patriots have been applying analytics to evaluate players and orchestrate roster management since 1994, when Robert Kraft bought the team from Jim Orthwein. Of course, 1994 was also the first year of the NFL salary cap and Jonathan Kraft, with his background in finance at Bain & Co., anticipated that cap management would become a key to team success. Together with his former workmate at Bain, Andy Wasynczuk, the Pats’ front office began to build statistical models for valuing players and roster management. When Bill Belichick became the assistant head coach in 1996, the Pats were able to begin to integrate the on-field and front office practices, and then were able to build upon this when Belichick returned to the Patriots as head coach in 2000. Belichick not only brought superb coaching skills but also a keen intellect that was able to understand and not be threatened by the statistical approach of the team’s front office. As time went on, Belichick integrated new game strategies based on statistical analysis. Unlike many coaches in professional sports, Belichick was willing to take heat from the media if he tried innovative game tactics and failed—knowing that the probabilities of success were on his side, even though his novel choices would not succeed every time. It is noteworthy that, notwithstanding Michael Lewis’s interpretation that the Oakland A’s around the turn of the century were the first team to marry new statistical analytical methods with the making of roster and strategy decisions, it appears that the Patriots may have beaten Billy Beane to the altar.

Analytical work on individual basketball performance began with Dave Heeren’s TENDEX system, as noted above. TENDEX was modified numerous times over the years, first dividing output by minutes played in the 1960s, then adding blocked shots, steals, and turnovers in the 1970s as the NBA
began to tabulate these stats, among other elaborations. TENDEX became the basis for a variety of linear weights models, each adding or subtracting various basketball plays and weighting each according to their estimated contribution to scoring or net scoring.

The simplest adaptation of TENDEX was the NBA’s own NBA Efficiency rating. The formula for NBA Efficiency (NBA-E) was:

NBA-E = points + rebounds + assists + steals + blocks – missed field goals – missed free throws – turnovers

Note that the formula gave equal weight to each of these actions. John Hollinger developed a variant known as PER (Player Evaluation Rating), and Dean Oliver brought more sophistication to the analysis by incorporating the concepts of possession and pace.

In order to understand the true efficiency of a player, it is necessary to know the cost as well as the benefit of his actions. As noted above, once possession was defined, it was possible to estimate the average points scored per possession and, hence, know the average cost of losing possession (e.g., a missed field goal shot or a turnover).
¹⁴
A player’s total output is also affected by how many possessions a team has during a game, which, in turn, is determined by how fast the game is played (pace). Just as outs per team are equal in baseball, possessions per team are equal in basketball.

Despite the advances in the linear weights approach, David Berri has argued that most of the player and team performance models, by underweighting lost possession, have overvalued scoring, at the expense of other actions, such as rebounds, assists, or steals.
¹⁵
Berri has also observed that statistical models of player salaries overvalued scoring as well; that is, owners generally paid high scorers more than their actual contribution to team wins warranted.
¹⁶
Hence, Berri suggested the presence of a market inefficiency—that is, GMs could build cheaper and better teams by placing more priority on players’ nonscoring attributes.
¹⁷

While these linear weights models have improved our understanding of the game and of how to evaluate players, they also come up short in a number of ways. There are many elements of defensive performance that they do
not capture, and they do not do justice to the interdependence of play: for instance, whether a player has a shot, and whether it is an easy or difficult shot, may depend on a good pick being set, a sharp assist being made, or poor defense being executed.

Accordingly, a more inclusive player evaluation approach was introduced. This approach, the plus/minus and adjusted plus/minus systems, was adapted from the plus/minus system in hockey. The plus/minus system in hockey simply records whether the team gained or lost net goals when a player was on the ice.
¹⁸
The system in basketball records the net gain or loss in points while a player is on the court. So, if the Miami Heat is behind 50–48 when LeBron James enters the game, but ahead 60–54 when he goes back to the bench, then James takes the team from down 2 to up 6, and he is credited with a +8 score for that period of play. By its nature, the plus/minus tally allows
all
events on the court to influence the recorded score attributed to the player and it does not have to be concerned with either identifying or weighting the various events.

One problem with the plus/minus system is that the presence of an individual player in the game may have no, little, or substantial impact on the score differential. Accordingly, the plus/minus model has been modified to account for the quality of the other players on the court—this is called the Adjusted Plus/Minus (APM). Even in this case, however, the synergy or lack thereof of a particular group of players will still influence a player’s APM. Further, from the point of view of a GM or a coach, the APM or raw plus/minus system tells one little about how the specific skills that a player brings to the court influence the observed score differential. Finally, the APM value shows low reliability (i.e., low correlation for individual players from one year to the next);
¹⁹
endowing the metric with little predictive value and making it problematic to use as a guide to personnel decisions.