measuring honesty, with p=.006…

Simon Gächter and Jonathan Schulz recently published a paper in Nature attempting to link intrinsic (individual) honesty with a measure of corruption in the subject home country. Out of more than 2,500 subjects in 23 countries. [I am now reading Nature on a regular basis, thanks to our lab subscribing a coffee room subscription!] Now I may sound naïvely surprised at the methodological contents of the paper and at a publication in Nature but I never read psychology papers, only Andrew’s rants at’em!!!

“The results are consistent with theories of the cultural co-evolution of institutions and values, and show that weak institutions and cultural legacies that generate rule violations not only have direct adverse economic consequences, but might also impair individual intrinsic honesty that is crucial for the smooth functioning of society.”

The experiment behind this article and its rather deep claims is however quite modest: the authors asked people to throw a dice twice without monitoring and rewarded them according to the reported result of the first throw. Being dishonest here means reporting a false result towards a larger monetary gain. This sounds rather artificial and difficult to relate to dishonest behaviours in realistic situations, as I do not see much appeal in cheating for 50 cents or so. Especially since the experiment accounted for a difference in wealth backgrounds, by adapting to the hourly wage in the country (“from $0.7 dollar in Vietnam to $4.2 in the Netherlands“). Furthermore, the subjects of this experiment were undergraduate students in economics departments: depending on the country, this may create a huge bias in terms of social background, as I do not think access to universities is the same in Germany and in Guatemala, say.

“Our expanded scope of societies therefore provides important support and qualifications for the generalizability of these theories—people benchmark their justifiable dishonesty with the extent of dishonesty they see in their societal environment.”

The statistical analysis behind this “important support” is not earth-shattering either. The main argument is based on the empirical cdfs of the gain repartitions per country (in the above graph), with tests that the overall empirical cdf for low corruption countries is higher than the corresponding one for high corruption countries. The comparison of the cumulated or pooled cdf across countries from each group is disputable, in that there is no reason the different countries have the same “honesty” cdf. The groups themselves are built on a rough measure of “prevalence of rule violations”. It is also rather surprising that for both groups the percentage of zero gain answers is “significantly” larger than the expected value of 2.8% if the assumption of “justified dishonesty” holds. In any case, there is no compelling argument as to why students not reporting the value of the first dice would naturally opt for the maximum of the two dices. Hence a certain bemusement at this paper appearing in Nature and even deserving an introductory coverage in the first part of the journal…