16/6/17: Replicating Scientific Research: Ugly Truth

Continuing with the theme on ‘What I’ve been reading lately?’, here is a smashing paper on ‘accuracy’ of empirical economic studies.
The paper, authored by Hou, Kewei and Xue, Chen and Zhang, Lu, and titled “Replicating Anomalies” (most recent version is from June 12, 2017, but it is also available in an earlier version via NBER) effectively blows a whistle on what is going on in empirical research in economics and finance. Per authors, the vast literature that detects financial markets anomalies (or deviations away from the efficient markets hypothesis / economic rationality) “is infested with widespread p-hacking”.
What’s p-hacking? Well, it’s a shady practice whereby researchers manipulate (by selective inclusion or exclusion) sample criteria (which data points to exclude from estimation) and test procedures (including model specifications and selective reporting of favourable test results), until insignificant results become significant. In other words, under p-hacking, researchers attempt to superficially maximise model and explanatory variables significance, or, put differently, they attempt to achieve results that confirm their intuition or biases.
What’s anomalies? Anomalies are departures in the markets (e.g. in share prices) from the predictions generated by the models consistent with rational expectations and the efficient markets hypothesis. In other words, anomalies occur when markets efficiency fails.

This post was published at True Economics on Friday, June 16, 2017.