[op-ed snap]A short history of data

Note4Students

Mains Paper 2: Governance | Government policies and interventions for development in various sectors and issues arising out of their design and implementation.

From UPSC perspective, the following things are important:

Prelims level: Basic knowledge of India’s data regime.

Mains level: The news-card analyses issues and challenges with India’s data regime, in a brief manner.


NEWS

CONTEXT

In January, two independent members of the National Statistical Commission resigned in protest, over alleged suppression of economic data by the government.

Past glory of statistical governance

  • While declining data quality has been an issue for a while, concern over institutional independence is new.
  • What several of these criticisms reference is the fact that India’s national statistics were once internationally renowned among economists and policy professionals for their reliability.
  • In the decades following World War II, India had reason to be proud not only of the institutional independence of national statistical bodies but also — uniquely among developing countries — of a pioneering history of independent data collection and publication.
  • The growth of India’s vast national statistical infrastructure dates back to its first decade as an independent country.
  • The birth of a new nation led to an explosion of national statistics, based on the need to plan the economy through Five Year Plans.
  • These years would see the establishment of the office of the Statistical Adviser to the Government, bi-annual National Sample Surveys (NSS), the Central Statistical Organisation (CSO), and National Income Committees (that made the estimates similar to GDP measurements).
  • The moving spirit behind these developments was Prasanta Chandra Mahalanobis, whom Jawaharlal Nehru described as the “presiding genius of statistics in India,” and the institute that he had founded in Calcutta in 1931, the Indian Statistical Institute (ISI).
  • ‘The Professor’, as Mahalanobis was known to associates, was involved in the discussions that led to establishment of the UN Statistical Commission in New York (a body that he would be voted Chairperson of several times during the 1950s).
  • As a pioneer in the emerging field of large-scale sample surveys, he would also be the force behind creating the UN Sub-Commission on Statistical Sampling in 1947, co-authoring the textbook on the subject in 1950.
  • The Institute’s fingerprints were readily apparent in the creation of India’s National Income Committee, the Central Statistical Organisation, the International Statistical Education Centre in Calcutta, and the National Sample Survey — all created around the mid-century mark.

The effectiveness of National Sample Survey

  • The results of the National Sample Survey offered high-definition snapshots of the country’s material life — casting light on cost of living, crop estimates, household consumption, industry, trade, and land holding patterns.
  • Twenty years later, the once sceptical Edwards Deming was now a convert: “No country, developed, under-developed or over-developed, has such a wealth of information about its people as India.”
  •  The contemporary Singaporean statistician Y.P. Seng observed that by comparison that China had “no genuine statistics” and so India’s example of using surveys would “serve as a guide and an example worthy of imitating”.
  • With their combined influence on the UN Statistical Commission and the UN Sub-Commission on Statistical Sampling, the Indian Statistical Institute and the National Sample Survey continue to have a lasting impact on estimating poverty across the developing world.
  • Methods pioneered by the National Sample Survey have become the norm for household surveys across the globe. For example, the Living Standard Measurement Study surveys conducted in several countries by the World Bank can trace their lineage back to the work of Indian statisticians associated with the Indian Statistical Institute and the National Sample Survey.

Conclusion

This distinguished history, which India can claim with pride, makes the recent undermining of the credibility of our statistical output especially regrettable. We can, however, ensure that when we look back on this several years from now, it represents an anomaly rather than a lasting, irreparable loss of institutional credibility.