At first glance, Paying Star Employees Well is a Good Strategy for Innovation in a Stanford business school newsletter seems like one of those "no-doh" discoveries that happen when academics turn their focus on the real world and end up confirming the obvious. But I clicked, and read, and it turns out to be much more interesting than that. They researched what people and companies actually do, not just what they say.

The article admits that the research conclusion is obvious:

Is there a link between the market strategies of software companies and how much they pay their employees, particularly the best and the brightest? There is. Software firms competing in volatile sectors in which the potential payoff for innovative products is the greatest — and the penalty for failure is severe — pay more to star workers than companies in more stable segments of the industry.

And then gets to the much more interesting topic of how they researched it:

The paper is noteworthy for more than its insight into the software industry. It’s an example of how insider econometrics, the careful analysis of rich, new sources of data, combined with interviews of industry insiders, is leading to a much clearer understanding of the efficacy of management strategies in general, and human resources practices in particular.

Now that’s intriguing. I dislike research that asks people questions and codifies their collective answers. Most of the time the people asked are a skewed list. And then there’s a huge gap between what people say and what they think, and another huge gap between what they think and what they actually do.

In this case, though, they studied what companies actually pay, and to whom. None of this is based on polling people and analyzing answers:

Testing that assumption would have been difficult in the past. Detailed data on a broad sampling of employees was simply not available, which is why most compensation studies until recently have focused on CEO pay. Empirical studies, say the researchers, had yet to establish a link between product market strategy and human resource practices using data covering more than a small number of firms or a select group of employees.

Now, however, much broader and deeper data is available through the Longitudinal Employer-Household Dynamics Program of the U.S. Census Bureau. The researchers analyzed salary and revenue data from software firms in 10 states, including exercised stock options and bonuses. Using that data, they were able to calculate the different potential payoffs for various types of software products and how those payoffs related to compensation.

Not only was there more data available for use in this study, it’s of better quality. For example, many studies that use income information derive it from surveys or interviews, while the "Stars" study used unemployment insurance data to find out exactly what people were paid over a period of time, rather than relying on their memories.

The researchers needed to understand the range of payoffs of software projects. To do this, they used firm-specific data for companies in 30 software sectors from the Economic Census. Ultimately, they created a "product payoff dispersion" measure for each company, which in turn created a model of project selection.

The research did eventually get into talking to real people in the field, asking their opinions. But even that phase wasn’t a poll, but rather a series of in-depth interviews, which I think (hope) means that researchers were able to get through the objectivity of simple polling and look into what people really do.

Conclusion: research is good, data is good, but knowing what people actually do is way better than just knowing how a group of people answers a set of questions.