This week, rather extraordinarily!, Nature book review was about official statistics, with a review of Julia Lane’s Democratizing our Data. (The democratizing in the title is painful to watch, though!) The reviewer is Beth Simone Noveck, who was deputy chief technology officer under Barack Obama and a major researcher in digital democracy, excusez du peu! (By comparison, Trump’s deputy chief technology officer had a B.A. in politics and no other qualification for the job, but got nonetheless promoted to chief…)
“Lane asserts that the United States is failing to adequately track its population, economy and society. Agencies are stagnating. The census dramatically undercounts people from minority racial groups. There is no complete national list of households. The data are made available two years after the count, making them out of date as the basis for effective policy making.” B.S. Noveck
The debate raised by the book on the ability of official statistics to keep track of people in a timely manner is most interesting. And not limited to the USA, even though it seems to fit in a Hell of its own:
“In the United States, there is no single national statistical agency. The process of gathering and publishing public data is fragmented across multiple departments and agencies, making it difficult to introduce new ideas across the whole enterprise. Each agency is funded by, and accountable to, a different congressional committee. Congress once sued the commerce department for attempting to introduce modern techniques of statistical sampling to shore up a flawed census process that involves counting every person by hand.” B.S. Noveck
This remark brings back to (my) mind the titanesque debates of the 1990s when Republicans attacked sampling techniques and statisticians like Steve Fienberg rose to their defence. (Although others like David Freedman opposed the move, paradoxically mistrusting statistics!) The French official statistic institute, INSEE, has been running sampled census(es) for decades now, without the national representation going up in arms. I am certainly being partial, having been associated with INSEE, its statistics school ENSAE and its research branch CREST since 1982, but it seems to me that the hiring of highly skilled and thoroughly trained civil servants by this institute helps in making the statistics it produces more trustworthy and efficient, including measuring the impact of public policies. (Even though accusations of delay and bias show up regularly.) And in making the institute more prone to adopt new methods, thanks to the rotation of its agents. (B.S. Noveck notices and deplores the absence of reference to foreign agencies in the book.)
“By contrast, the best private-sector companies produce data that are in real time, comprehensive, relevant, accessible and meaningful.” B.S. Noveck
However, the notion in the review (and the book?) that private companies are necessarily doing better is harder to buy, if an easy jab at a public institution. Indeed, public official statistic institutes are the only one to have access to data covering the entire population, either directly or through other public institutes, like the IRS or social security claims. And trusting the few companies with a similar reach is beyond naïve (even though a company like Amazon has almost an instantaneous and highly local sensor of economic and social conditions!). And at odds for the call of democratizing, as shown by the impact of some of these companies on the US elections.
Xi'an's Og 