More on Wages of Wins

Lots of folks, including True Hoop and Slam online, have followed the recent debate between the authors of The Wages and Wins (hereafter WOW) and John Hollinger, about the best ways to measure the value of NBA players. There appear to be two major sources of contention - one about the relative value of a rebound, and the other about the relative harm of a missed shot. About the former, without getting into all the details, it seems that both approaches, though they’ve been arguing about it, put a similar value on rebounding. About the latter, the differences of opinion are more obvious.

You can follow the links to wade through the discussion, in which the New Yorker’s Malcolm Gladwell, who favorably reviewed The Wages of Wins, gets involved.

I am not a statistician, though I sometimes play one on this blog. But, as I was reading through the latter half of the Wages of Wins book last night (this is what I do when I can’t sleep at 3 am), I had some thoughts about WOW's approach to evaluating player performance in the NBA that I wanted to take a run at.

As I mentioned above, the two key issues here are rebounding and offensive effiicency, or missed shots. There's a larger issue that connects both these things, which I will explain. But, let me take on the two key issues in order.

Rebounding (or, the David Ortiz vs. Dennis Rodman problem)

The foundation of the analysis in WOW is to first ask: which outcomes on a basketball court best correlate with wins. The authors don’t care whether people think scoring is most important, or rebounds, or whatever. They want to analyze the statistical relationships between those outcomes and winning. That is the necessary starting point for asking: which player is the best player – that is, the player who contributes most to his team’s winning. This is the right place to start. David Berri, one of the WOW authors, criticizes Hollinger for failing to build from this foundation and, in particular, for failing to use a rigorous statistical technique (regression analysis) to figure out the relationship between a specific act on the court (a steal, or a missed shot) and a team’s likelihood of winning a game.

Once the authors know what the components of winning and losing are, they can begin to construct a formula (or formulas) that assess the contribution of individual players to winning. If getting rebounds, and avoiding missed shots, and scoring, and steals, and not fouling have the highest correlation with success at the team level, then we can look at the individuals who do those things the best and quantify, reasonably well, how much the individual players, by achieving more of those positive outcomes, contribute to team success.

So, in short, the authors are right, I think, to start by figuring out what explains wins and losses at the team level and working backwards from there to evaluate individual contributions to team success. But, what I think they've missed is properly accounting for the team nature of the game. I won’t bore you with talk about regression analysis and variable independence, but I will say this – for the WOW approach to work in basketball, they must assume that one act by one player doesn’t really affect the same act by another player on the same team. So, for example, if you believe that rebounds lead to wins, you can then start evaluating how good individual players are by how many rebounds they get. But, only to a point. Why? Because maybe one player on a team will get a lot of rebounds because that’s part of his role and maybe, if he’s especially good at it, other players on his team will accommodate that skill and think of something else to do, like engage in more weak side help on defense, or challenging shots on the perimeter, or releasing early on the fast break. The authors themselves note that when Dennis Rodman joined the Bulls for the 1995-96 season, one effect of his presence on the team was likely to suppress the rebounding totals for other players on his own team.

This matters for their analysis for reasons which I’ll explain below. But, first I want to note that this is very different from evaluating baseball statistics, particularly offensive ones. The fact that David Ortiz might hit 48 homers in a season has relatively little to do with 2006 Red Sox teammate Alex Gonzalez’ home run totals. The hitters in front of Big Papi may try a little bit to focus on getting on base, rather than hitting homeruns because Papi’s coming up behind them, but the reality is that they are, for the most part, going to approach hitting the way they would in any event. And, for players farther down in the lineup, there should be no effect at all. Jason Varitek isn’t going to watch Ortiz hit a homerun and decide he doesn’t need to do that. He’s going to try to hit the way he would in any circumstance, in all likelihood.  Perhaps some caveats need to be made for guys batting directly in front of or behind Ortiz, but the performance of players in the Sox’ lineup can plausibly be said to be independent of performance by others in the same lineup.

In my judgment, you simply can’t say this about basketball. On offense, Baseball is a sequential game. Basketball is an interdependent one. The difference is, I think, of profound consequence for whether you can disentangle individual performance from team performance. To repeat, in baseball, I think you can do this, at least for offense, for the most part. But, I am not convinced that you can do this for basketball with nearly the same confidence.

Let’s go back to Rodman. A key question that, as far as I can tell, is not answered by the WOW authors is this: what is the difference between the number of rebounds the Bulls would have secured with a replacement for Rodman compared to the number they did secure with Rodman? The answer, I am sure is “fewer.” But, how many fewer? If a Rodman replacement snared seven rebounds a game, instead of 15, can we really say that the switch cost the team eight rebounds a game. I don’t think so, and neither, I suspect, do the authors. But, again, I want to emphasize, this is different from baseball. If the Sox replace Ortiz, a forty-five homerun hitter, with a 25 homerun hitter, I think it’s a fair assumption that the Sox are going to lose about twenty homeruns. It’s unlikely that other players on the Sox are going to hit more homeruns in order to compensate for Ortiz’ absence, because homerun hitting (and nearly every other offensive skill in baseball) isn’t subject to that kind of willful decision-making. You can either do it, or you can’t. Baseball offenses are sequential, not interdependent. And, this changes the nature of the evaluation of individual statistics.

I agree that rebounds contribute to winning, but to know how much Rodman contributed to winning, we need to know more than how many rebounds he got and we need to know more than how many rebounds he got relative to players on other teams.. What we really need to know is how many rebounds his own team got that it would not otherwise have gotten if he weren’t there. Do I have a clue how to figure that out? Absolutely not. But, I am convinced that that is the key question and that it has not been answered by WOW (and, in fairness, other similar systems).

So, that’s my problem with the way the WOW authors evaluate rebounding (and basketball stats more generally). Before I move on to missed shots, let me make a final point here. Above, I noted that with a proficient rebounder like Rodman (or Ben Wallace) on your team, maybe teammates elect to do other things (or are told by their coaches to do other things). And, those other things I mentioned, like giving more help on defense, or releasing on the break, could be an unmeasured benefit of the presence of a Rodman or a Wallace. But, my point remains that the move from team-level correlates of winning to individual performance is a problematic one in basketball. Missed shots illustrate this same point.

Shooting Efficiency (or, the Juan Pierre vs. Allen Iverson problem)

The other key issue concerns shooting efficiency or, in plain English, missed shots. In baseball, Juan Pierre makes a lot of outs. We can all agree that making outs suppresses offense. Given Pierre’s propensity to make outs (the flip side of his relatively weak on-base skills and the fact that he normally bats lead-off so he gets a lot at bats), we can say that this propensity hurts the offense relative to another player who makes fewer outs.

In basketball, Allen Iverson takes and misses a lot of shots and, because of that, the WOW system regards him as something short of a great player (and, mind you, they may be right). But, can we really say the same about Iverson’s missing shots as we can about Pierre’s making outs? I don’t think we can. Missing shots is bad. But, remember, just as basketball is inter-dependent whereas baseball is sequential (at least on offense), hoops are also closed-ended, whereas baseball is open-ended. In basketball, you can’t just hold the ball all day and wait for the perfect shot. You must shoot within 24 seconds. By contrast, teams in baseball are not required to make outs. Theoretically, they could keep hitting all day if their players could keep avoiding making outs. So, if Iverson is on a team of relatively limited offensive players and, in a given possession, he’s more likely than not their best option to shoot, which they must do at some point, he may take and miss a lot of shots. But, this fact, while not uninformative, tells us too little about the context in which those shots are going up, as I’ll explain in a moment. By contrast, if Pierre is making lots of outs, there is a less plausible contextual explanation for that deficiency – it’s likely because Pierre is just not good at avoiding them.

The goal of every offense in basketball is to convert possessions into points. About this fundamental fact, Berri et al. and Hollinger agree. So, even once we’ve established that missing shots is bad for a team, it’s still not enough to know merely whether Iverson misses a lot of shots in evaluating his individual contribution to team wins. Instead, what we really need to know is this - whether, with another player, Philadelphia would convert a higher percentage of its possessions into points than they do with Iverson. I am not sure of the answer to that, but I see no evidence that it’s been addressed. And, since it hasn’t, I think there are reasons to believe that AI is being undervalued. As Hollinger notes, equating a missed shot with a turnover is flawed because a missed shot can be rebounded, while a turnover can’t (In Hollinger's Player Efficiency Ratings, missed shots are not regarded as equally negative to a turnover - in the WOW system, they are). In other words, a missed shot may be less bad than the alternative.

The larger point here is that missed shots are being evaluated as if they occurred in a vacuum. We know that Pierre isn’t making outs to somehow otherwise benefit the team. He’s making outs because he’s not good at getting on base. Iverson may be missing shots because, frankly, he has no choice. And, surrounded by the same talent, and given the same responsibilities, a player who shoots a higher percentage on another team might come to Philly and perform worse than Iverson, while freeing up fewer opportunities for his teammates than Iverson might.

I want to emphasize that I don’t know how Iverson would rate according to a more contextualized approach. But, what I am quite sure of is that the underlying logic here, of extrapolating from the building blocks of winning on the team level to individual player performance is far more problematic in basketball than in baseball andgives me pause about its efficacy.

WOW values players like Rodman and Wallace especially highly. And, I love both players and the unique skill sets they’ve brought to the NBA. But, I suspect that their aversion to shooting is a detriment to their teams in ways not captured by the WOW system. The fact that they take relatively few shots may be interpreted to mean that they’re minimizing their own offensive inefficiency. But, if the consequence of their presence is that their team is playing four on five, the effects of Rodman or Wallace’s presence on the team’s efficiency may be detrimental in ways not captured by Rodman’s or Wallace’s individual statistics. This is one reason why I think Hollinger is right to give some credit for taking shots since, the fact is, somebody has to at some point.

The success of the teams Rodman and Wallace have played on suggests their positive impact. But, there’s an unfilled gap here between individual performance and the correlates of team success that cannot be closed as readily in basketball as in baseball. I think it’s a fairly significant hole in the analysis, though I still love the book.