Big Data, Financial Inclusion and Privacy for the Poor

Guest Post by Dr Katharine Kemp, Research Fellow, UNSW Digital Financial Services Regulation Project

Financial inclusion is not good in itself.

We value financial inclusion as a means to an end. We value financial inclusion because we believe it will increase the well-being, dignity and freedom of poor people and people living in remote areas, who have never had access to savings, insurance, credit and payment services.

It is therefore important to ensure that the way in which financial services are delivered to these people does not ultimately diminish their well-being, dignity and freedom. We already do this in a number of ways – for example, by ensuring providers do not make misrepresentations to consumers, or charge exploitative or hidden rates or fees. Consumers should also be protected from harms that result from data practices, which are tied to the provision of financial services.

Benefits of Big Data and Data-Driven Innovations for Financial Inclusion

“Big data” has become a fixture in any future-focused discussion. It refers to data captured in very large quantities, very rapidly, from numerous sources, where that data is of sufficient quality to be useful. The collected data is analysed, using increasingly sophisticated algorithms, in the hope of revealing new correlations and insights.

There is no doubt that big data analytics and other data-driven innovations can be a critical means of improving the health, prosperity and security of our societies. In financial services, new data practices have allowed providers to serve customers who are poor and those living in remote areas in new and better ways, including by permitting providers to:

  • extend credit to consumers who previously had to rely on expensive and sometimes exploitative informal credit, if any, because they had no formal credit history;
  • identify customers who lack formal identification documents;
  • design new products to fit the actual needs and realities of consumers, based on their behaviour and demographic information; and
  • enter new markets, increasing competition on price, quality and innovation.

But the collection, analysis and use of enormous pools of consumer data has also given rise to concerns for the protection of financial consumers’ data and privacy rights.

Potential Harms from Data-Driven Innovations

Providers now not only collect more information directly from customers, but may also track customers physically (using geo-location data from their mobile phones); track customers’ online browsing and purchases; and engage third parties to combine the provider’s detailed information on each customer with aggregated data from other sources about that customer, including their employment history, income, lifestyle, online and offline purchases, and social media activities.

Data-driven innovations create the risk of serious harms both for individuals and for society as a whole. At the individual level, these risks increase as more data is collected, linked, shared, and kept for longer periods, including the risk of:

  • inaccurate and discriminatory conclusions about a person’s creditworthiness based on insufficiently tested or inappropriate algorithms;
  • unanticipated aggregation of a person’s data from various sources to draw conclusions which may be used to manipulate that person’s behaviour, or adversely affect their prospects of obtaining employment or credit;
  • identity theft and other fraudulent use of biometric data and other personal information;
  • disclosure of personal and sensitive information to governments without transparent process and/or to governments which act without regard to the rule of law; and
  • harassment and public humiliation through the publication of loan defaults and other personal information.

Many of these harms are known to have occurred in various jurisdictions. The reality is that data practices can sometimes lead to the erosion of trust in new financial services and the exclusion of vulnerable consumers.

Even relatively well-meaning and law-abiding providers can cause harm. Firms may “segment” customers and “personalise” the prices or interest rates a particular consumer is charged, based on their location, movements, purchase history, friends and online habits. A person could, for example, be charged higher prices or rates based on the behaviour of their friends on social media.

Data practices may also increase the risk of harm to society as a whole. Decisions may be made to the detriment of entire groups or segments of people based on inferences drawn from big data, without the knowledge or consent of these groups. Pervasive surveillance, even the awareness of surveillance, is known to pose threats to freedom of thought, political activity and democracy itself, as individuals are denied the space to create, test and experiment unobserved.

These risks highlight the need for perspective and caution in the adoption of data-driven innovations, and the need for appropriate data protection regulation.

The Prevailing “Informed Consent” Approach to Data Privacy

Internationally, many data privacy standards and regulations are based, at least in part, on the “informed consent” – or “notice” and “choice” – approach to informational privacy. This approach can be seen in the Fair Information Practice Principles that originated in the US in the 1970s; the 1980 OECD Privacy Guidelines; the 1995 EU Data Protection Directive; and the Council of Europe Convention 108.

Each of these instruments recognise consumer consent as a justification for the collection, use, processing and sharing of personal data. The underlying rationale for this approach is based on principles of individual freedom and autonomy. Each individual should be free to decide how much or how little of their information they wish to share in exchange for a given “price” or benefit. The data collector gives notice of how an individual’s data will be treated and the individual chooses whether to consent to that treatment.

This approach has been increasingly criticised as artificial and ineffectual. The central criticisms are that, for consumers, there is no real notice and there is no real choice.

In today’s world of invisible and pervasive data collection and surveillance capabilities, data aggregation, complex data analytics and indefinite storage, consumers no longer know or understand when data is collected, what data is collected, by whom and for what purposes, let alone how it is then linked and shared. Consumers do not read the dense and opaque privacy notices that supposedly explain these matters, and could not read them, given the hundreds of hours this would take. Nor can they understand, compare, or negotiate on, these privacy terms.

These problems are exacerbated for poor consumers who often have more limited literacy, even less experience with modern uses of data, and less ability to negotiate, object or seek redress. Yet we still rely on firms to give notice to consumers of their broad, and often open-ended, plans for the use of consumer data and on the fact that consumers supposedly consented, either by ticking “I agree” or proceeding with a certain product.

The premises of existing regulation are therefore doubtful. At the same time, some commentators question the relevance and priority of data privacy in developing countries and emerging markets.

Is data privacy regulation a “Western” concept that has less relevance in developing countries and emerging markets?

Some have argued that the individualistic philosophy inherent in concepts of privacy has less relevance in countries that favour a “communitarian” philosophy of life. For example, in a number of African countries, “ubuntu” is a guiding philosophy. According to ubuntu, “a person is a person through other persons”. This philosophy values openness, sharing, group identity and solidarity. Is privacy relevant in the context of such a worldview?

Privacy, and data privacy, serve values beyond individual autonomy and control. Data privacy serve values which are at the very heart of “communitarian” philosophies, including compassion, inclusion, face-saving, dignity, and the humane treatment of family and neighbours. The protection of financial consumers’ personal data is entirely consistent with, and frequently critical to, upholding values such as these, particularly in light of the alternative risks and harms.

Should consumer data protection be given a low priority in light of the more pressing need for financial inclusion?

Some have argued that, while consumer data protection is the ideal, this protection should not have priority over more pressing goals, such as financial inclusion. Providers should not be overburdened with data protection compliance costs that might dissuade them from introducing innovative products to under-served and under-served consumers.

Here it is important to remember how we began: financial inclusion is not an end in itself but a means to other ends, including permitting poor and those living in remote areas to support their families, prosper, gain control over their financial destinies, and feel a sense of pride and belonging in their broader communities. The harms caused by unregulated data practices work against each of these goals.

If we are in fact permanently jeopardising these goals by permitting providers to collect personal data at will, financial inclusion is not serving its purpose.


There will be no panacea, no simple answer to the question of how to regulate for data protection. A good starting place is recognising that consumers’ “informed consent” is most often fictional. Sensible solutions will need to draw on the full “toolkit” of privacy governance tools (Bennett and Raab, 2006), such as appropriate regulators, advocacy groups, self-regulation and regulation (including substantive rules and privacy by design). The solution in any given jurisdiction will require a combination of tools best suited to the context of that jurisdiction and the values at stake in that society.

Contrary to the approach advocated by some, it will not be sufficient to regulate only the use and sharing of data. Limitations on the collection of data must be a key focus, especially in light of new data storage capabilities, the likelihood that de-identified data will be re-identified, and the growing opportunities for harmful and unauthorised access the more data is collected and the longer it is kept.

Big data offers undoubted and important benefits in serving those who have never had access to financial services. But it is not a harmless curiosity to be mined and manipulated at the will of those who collect and share it. Personal information should be treated with restraint and respect, and protected, in keeping with the fundamental values of the relevant society.



Colin J Bennett and Charles Raab, The Governance of Privacy (MIT Press, 2006)

Gordon Hull, “Successful Failure: What Foucault Can Teach Us About Privacy Self-Management in a World of Facebook and Big Data” (2015) 17 Ethics and Information Technology Journal 89

Debbie VS Kasper, “Privacy as a Social Good” (2007) 28 Social Thought & Research 165

Katharine Kemp and Ross P Buckley, “Protecting Financial Consumer Data in Developing Countries: An Alternative to the Flawed Consent Model” (2017) Georgetown Journal of International Affairs (forthcoming)

Alex B Makulilo, “The Context of Data Privacy in Africa,” in Alex B Makulilo (ed), African Data Privacy Laws (Springer International Publishing, 2016)

David Medine, “Making the Case for Privacy for the Poor” (CGAP Blog, 15 November 2016)

Lokke Moerel and Corien Prins, “Privacy for the Homo Digitalis: Proposal for a New Regulatory Framework for Data Protection in the Light of Big Data and the Internet of Things” (25 May 2016)

Office of the Privacy Commissioner of Canada, Consent and Privacy: A Discussion Paper Exploring Potential Enhancements to Consent Under the Personal Information Protection and Electronic Documents Act (2016)

Omri Ben-Shahar and Carl E Schneider, More Than You Wanted to Know: The Failure of Mandated Disclosure (Princeton University Press, 2016)

Productivity Commission, Australian Government, “Data Availability and Use” (Productivity Commission Inquiry Report No 82, 31 March 2017)

Bruce Schneier, Data and Goliath: The Hidden Battles to Collect Your Data and Control Your World (WW Norton & Co, 2015)

Daniel J Solove, “Introduction: Privacy Self-Management and the Consent Dilemma” (2013) 126 Harvard Law Review 1880


Agricultural Markets: Five Opportunities for Innovation After Demonetisation


Guest post by Samir Shah, MD & CEO, NCDEX

Due to the demonetisation of currency and recall of the currency notes of Rs. 500 and Rs. 1000 denominations announced by the government there was some panic in the initial days and the mandis were closed for some days. However, things are becoming better with the passage of time.

Recent visits to the mandis and fresh feedback received from the market participants indicates that arrivals have started improving in the mandis. The Rabi sowing status, which was feared to be disrupted due to cash crunch, has not seen any major impact as most of the essential inputs were available on credit or in old currency notes.

In the weeks after demonetisation, it was observed that commercial small farmers in “tight” value chains (such as sugarcane delivered to a specific neighbouring sugar mill, fruits, vegetables, milk, tea, specialty coffee and spices, fertilisers, seeds) benefitted from the strong relations between buyers/traders and producers and quickly adopted digital payment. These tight value chains generally involve greater control of the flow of goods and funds to ensure repayment (via delivery of the crop) and limit the opportunities of side selling (when the farmer delivers somewhere else to avoid repayment of loans extended under value chain financing models).

Government policies that support and encourage such value chains will speed up the shift to formal bank financing, financing from buyers (e.g., sugar mills, cotton ginners, milk companies), increase financial literacy and understanding about banking requirements among small farmers.

In looser supply chains where crops can be sold on the side and where repayment is difficult to capture through delivery, local lenders who are near farmers have an advantage, as proximity closes the asymmetric information gap, facilitates credit assessment, and makes repayment enforcement easier. Government policies should use input suppliers in the area, local credit unions, credit cooperatives, and microfinance institutions (MFIs) in that location for reaching out to such farmers.

The current situation, therefore, warrants exploring and providing more user-friendly and easily accessible scale-neutral technology, which can serve the economic needs of India’s 138 million farms. Fortunately India has already developed and successfully tested some of the best farm market-based ecosystem in the form of online agricultural markets.

Digitisation is opening up new opportunities for cost-saving automation, accuracy, speed and vastly-improved efficiency in agricultural trade documentation, storage, finance, and risk management. Supported by the right policies and market infrastructure institutions, it can transform Indian agriculture’s financing models, risk mitigation models, and distribution models.

1) Re-launch Exchange-traded Forwards and Launch Options

Exchange-traded commodities have already demonstrated the advantages from digitisation through greater speed, transparency, global reach, accuracy and reduced cost. Exchange traded Forward contracts (permitted by the erstwhile FMC, since suspended by SEBI), futures, and options allow crop prices to be locked in prior to the actual delivery of the product.

Exchange traded forwards can bring to cash-less and traceable (in addition to reducing defaults and better quality based sales) systems the entire Rs 7 – 9 lakh crore agri produce in India every year and enable the entire value chain to adopt newer and more compliant ways of doing business, including government procurement.

Except in wheat and rice that have partial protection through government procurement, Indian farmers are buffeted by price volatility. The availability of options can be the ideal instrument for insuring their margins. Farmer producer companies and cooperatives can be encouraged to use options to manage commercial risk in the production, processing and marketing of agricultural products. Banks can extend credit to purchase price insurance.

In other words, under the new agricultural market structure, farmers will be able to sell through transparent, digital markets such as the exchange-traded forwards or the National Agricultural Market/State Agricultural Market. They will also be able to sell to government agencies at the minimum support price through exchange-traded forwards. And they will be able to protect themselves from price volatility by using options.

Similarly, call options – that give the government the right – but not the obligation – to buy pulses when prices rise, for example, will reduce the need for accumulating physical stocks and add transparency by setting clear rules for government intervention. The food subsidy budget for FY17 is Rs 1.34 lakh crore, of which Rs 1.03 lakh crore is to be routed through FCI to the intended beneficiaries.

Exchanges can thus become the fulcrum of the new cashless agricultural economy if they move upfront on the developmental agenda of policymakers, regulator and political agencies.


2) Create more capacity in commodity exchanges in order to encourage more agricultural value-chain participants to use regulated markets for risk management and financing.

Agricultural sector companies not involved in primary production (i.e., traders, processors, food companies, input suppliers) have their own financing, and production and price risk management needs. Working capital, funding for acquisition of assets (movable and real estate), cash flow management services, hedging and insurance are often needed by these agricultural companies.

Increasing position limits, adding many more commodities for trade (such as pulses, rice and dairy), and reducing taxes (such as CTT) will encourage them to increasingly use the exchange platform for low-cost risk management, marketing, and inventory financing.

3) Expand the digital mandi network by connecting National Agricultural Market and State Agricultural Markets (SAM)

Early indications are that although NAM has been implemented in 250 mandis across India, it is still currently restricted to post-trade data entry and not functioning as real price discovery, clearing and settlement driven markets.

On the other hand, SAM initiatives in Karnataka have implemented e-trading virtually end to end. Rashtriya e-Market Services Limited (ReMS), a joint venture of Karnataka Government and NCDEX Spot Exchange Limited (NSPOT) was formed to setup a Unified Marketing Platform (UMP) for modernising more than 300 APMC regulated market yards into a single online marketplace for the state, and enhancing the efficiency of regulated markets in the state. The Karnataka Government provided unified licenses for all APMCs within Karnataka. It also allowed for warehouse-based sales, warehouse receipt-linked loans and single point levy of market fees across the state, making Karnataka one of the first few states to adopt all recommendations of model APMC Act. More than 66 lakh farmers have successfully completed transactions worth Rs 32,000 crore till date through the Unified Market Platform in Karnataka. A simple trading platform for “Tur” pulses trading in Gulbarga Mandi was provided and eventually government provided an e-trading UMP across the state. Through the e-platform, the state Government and NCDEX have played a key role in linking smallholders to a wider market and helping them realize better prices for their produce.

A study covering impact assessment of e-tendering of agricultural commodities in Karnataka conducted by National Institute of Agricultural Marketing, Government of India, reveals that about 83% of stakeholders felt that the operations have become more transparent and time-efficient. Farmers have reported an 18% increase in income realization and traders have reported at least 25% of their time being saved through the online process. Overall, all the mandis have experienced an increase in trading volume and revenue because of increased sale at the higher side of price range and stable prices.

Creating interoperability between NAM and SAM will help accelerate digitisation of mandis. Digitalisation is an important step for reducing inefficiencies in agricultural markets, developing rural financial services, transparent pricing and promoting better organised agricultural value chains. By connecting the two networks through interoperability to establish a comprehensive market architecture, farmers in almost 500 market yards can seamlessly experience the benefits of digitisation.



4) New WDRA-regulated repository will boost warehouse-based sales and commodity finance

Agricultural warehousing accounts for 15% of the warehousing market in India and is estimated to be worth Rs 8,500 crore.

Beginning can be made by exempting the 450 exchange-accredited warehouses, with a combined capacity of 2 million tonnes, from Stock Control Order under the Essential Commodities Act to encourage inflow of crops in this transparent and regulated warehousing network.

Simultaneously, the WDRA can encourage a pan-India digital network through the new Repository of all licensed warehouses for real-time data collection on food stocks. The benefits of switching to electronic accounting are almost immediate and lie in speed, ease of use, accuracy and cost. Automation reduces overheads and man-hours, with document transmission constrained only by the speed of the Internet.

The commodity repository will provide the legal and regulatory environment for inventory financing and warehouse receipt lending to encourage the use of these financing mechanisms. While currently the size of the market is estimated at about Rs 30,000 crore, as per a recent study by NABCONS, the potential for finance against collateral of major agri commodities and fertilisers is Rs 1,66,234 crore.

The combination of repository, digital warehouses, digital mandis, and warehouse receipts will create a legal environment that ensures easy enforceability of the security, and makes warehouse receipts a title document. It will create a network of reliable and high-quality warehouses that are publicly available. It will introduce a system of licensing, inspection, and monitoring of warehouses. It will lead to the creation of a performance bond and banks that trust and use the system. It will encourage agricultural market prices that reflect carrying costs. It will reduce the threat of hoarding of essential commodities and ease raw material procurement. And it will promote well-trained market participants.



5) Strengthen producer organisations as important aggregators for delivering digitised financial and non-financial services to smallholder farmers.

More than 18,500 small and marginal farmers have successfully hedged their crops on NCDEX in the last 10 months through nine Farmer Producer Companies. By creating the right mechanisms, more such companies can be encouraged to connect to formal, regulated, cash-less markets. There is also the need to invest resources in capacity building for financial and managerial skills as well as improved corporate governance.

For small farmers, the advantages of joining a collective are direct access to a viable market (local, regional, global) for the end product; a clear, transparent pricing mechanism, a price that is attractive; shift away from mono-cropping low-value high-volume crops; avoiding overreliance on credit to purchase inputs; leveraging a competitive advantage in production, quality certifications, proximity to the end market; and, credibility of the buyer and trust among farmers via regular direct interaction between the buyer and the farmers.

There are already a number of NGOs and initiatives that work to strengthen farmer producer organisations, but a more conscientious effort and a bigger scale is needed.

Post-demonetisation innovation can improve the following constraints in the agricultural sector:

  1. Enable transparent, efficient market structure (e.g., through greater adoption of exchanges)
  2. Reduce administrative and distribution costs of food procurement (e.g., through options and exchange-traded forwards)
  3. Improve security of the collateral and cash flows (e.g., warehouse receipt financing, price hedging, insurance).
  4. Improve competitiveness of small and marginal farmers (eg. Farmer producer companies)
  5. Increase the share of formal finance in agricultural commodity markets.

Replug – Interview with Dr. Viral Acharya

The Government today appointed Dr. Viral Acharya, Professor of Economics at the New York University Stern School of Business, as RBI Deputy Governor for 3 years. We had the pleasure of interviewing Dr. Acharya few years ago on aggregation of risks in a financial system especially with respect to the Indian context. In the below video we share Dr. Acharya’s conversation with Bindu on the subject. You can read the full transcript of the conversation here.

Dr. Viral Acharya in conversation with Bindu Ananth


Insights on Public Data & Visualisation – In conversation with co-founders of “How India Lives”


In this blog post we share an interview with Avinash Celestine & Avinash Singh, co-founders of How India Lives. Covering an array of topics, they share their experiences and thoughts on handling public data and the nuances of data visualisation that has to be kept in mind when dealing with complex datasets.

Tell us about how “How India Lives” came into being and the gaps that the platform is trying to address?

How India Lives was founded by five senior journalists after we felt reliable and relevant data, especially demographic, is missing in India. Often, we used to spend tremendous amount of time to collect and clean the data. We started the venture three years back and it was incubated at the Tow-Knight Center at the City University of New York. We received a grant from Tow-Knight Foundation to kick start the operations. The vision is not to duplicate existing data products, but to create products where reliable data can be easily accessed.

hilHow India Lives is a platform to distribute all kinds of public data in a searchable form, aiming to help people make informed business and policy decisions. Currently, data-driven decision making by a range of users – marketers, researchers, retailers, governments etc is driven by customer profiles created on the basis of sample surveys conducted by a number of companies. While these surveys have their uses, we feel that public data, organised under the Census, the National Sample Survey and other sources, can dramatically improve the way companies understand their customers. Their coverage is comprehensive and highly granular (the Census, for example, provides data on hundreds of variables down to every village and ward in India) in a way that many private consumer-oriented surveys are not. Rich possibilities also reside in several datasets beyond the well-known ones. Some of these datasets can be used as proxies to answer questions related to understanding the consumer landscape or making public policy interventions.

However, to organize public data so as to be useful to users, requires a major effort. Much public data is scattered across different databases (in many cases, it is not even organised into a database), which don’t ‘talk’ to each other, or are in unusable formats (e.g. pdf). Our aim is to reorganize public data into a common, machine readable format, and in a way that users can search for data and compare data from disparate sources, for a single geography. We also aim for this platform to be visual in nature, capturing data via maps and other appropriate visualisations.

Howindialives.com is presently in beta version and is scheduled to become paid in early-2017. At that time, we will have 250 million data points, more than 5,500 metrics, and at least 600 data points for each of the 715,000 geographical locations in India.

In addition, we also offer data and technology consultancy services to companies, media outfits and non-profits. Some of our clients: Mint, HT Media, Confederation of Indian Industry (CII), Daksh India, CBGA India, Centre for Policy Research (CPR), ABP News, TARU Leading Edge, Arghyam, and Founding Fuel.

A lot of our common understanding of government programs and other public initiatives is primarily driven by the data points that media dailies choose to write on especially in the context of large datasets. Given your experience as a journalist across media entities, what is your take on this and is there a room for improvement?

When media covers the release of new data, the coverage is often superficial, and unable to take into account the complexity of a dataset. The classic example here is the census. The census has been releasing data since 2012 or so. Often we have found that when these data releases are covered, it is only to the extent of state level data or national level data.

We feel that this is insufficient to take into account the vast geographical complexities of a country such as India. Often we have found that drilling down to a greater depth (e.g. down to district level), gives us greater insights, since disparities within states can often be as dramatic as those between states (e.g. see this link and the below map for an example of how, on one measure, disparities within states are often equally important ). This is just one example. Another way is that relationships between variables and how one variable can ‘cut’ another and to explore interrelationships between them. Our exploration of differing education levels among dalit castes is an example.

Another weakness is simply a lack of awareness of what is out there and to think creatively about what datasets can be used to address a particular question.

Then there is the problem that many datasets, while publicly available, are difficult to access – e.g. they may be in pdf format and/or scanned images. For instance, we explored the relationship between the real estate expansion in Gurgaon and the political regime, using a dataset that was in the public domain, but in PDF form, ensuring that it was unlikely to be used by journalists.

It’s important to stress that these weaknesses are not necessarily due to the incapacity of journalists themselves – indeed if that was the main problem it could be more easily addressed through training etc. The problem is deeper and is related to the way in which many media organisations work. Because of the pressure of deadlines, the need to publish on a regular basis, and the extremely short cycle of news, most journalists simply don’t have the time to be able to spend time with a dataset and understand its complexity (this is true of all news reporting – not just ‘data journalism’). The role of senior editors in giving their journalists the time they need to report and explore a piece of data, and insulating them to an extent from the daily news cycle, is crucial.

How would you characterize the demand for Data Visualization driven Journalism as opposed to the more traditional forms such as anecdotal evidence and story-telling?

The demand for data visualisation – journalism is certainly high, and it’s driven by the increasing availability of large datasets, and the tools to explore and visualise them (e.g. R for data analysis and d3 for javascript driven visualisations). These are supply-side factors. The proliferation of online news outlets, social media etc has provided the demand-side push.

Given the vast volumes of data that is now both available and accessible, data visualisation is increasingly becoming an ideal mode to digest and understand such data. In your experience of having worked on large datasets and visualised the same, what according to you are the fundamentals of a good visualization effort? Are there any nuances that someone has to keep in mind while balancing breaking down complex data and its visual presentation?

The best data journalism is one that combines all forms of story-telling and does not restrict itself to any one. Good data journalism is one that explores a dataset, covers the views of people who understand the field or area relevant to that data, and makes it clear what the data can and cannot tell us. (An excellent example of reporting that does this is the series done in The Hindu on rape cases )

Note : Also see answer to following question

When visualising a large dataset, the decision to not focus on certain data points is equally important as choosing what to. Is there a method or an optimum way that one can make these and/or choices? How much of this decision is based on the target audience and the end impact you want to have, say, on public policy for instance.

Any good data visualisation is driven by a clear viewpoint, developed from exploring the data and an iterative process between posing broader questions and seeing what the data throws up. Simply putting the data out there and assuming that it ‘speaks for itself’ as many claim, will almost guarantee that people’s engagement with it will be low.

The argument is often made that ‘imposing’ your viewpoint on the data is a no-no since it introduces bias. This ignores the fact that the very act of selecting how the data is to be shown and what to display and not to display, introduces bias anyway (otherwise we could just dump a giant excel file on users and ask them to figure it out for themselves). It’s better to take a clear line on what your data shows, and make your assumptions and line of argument clear. Readers are often smart enough to reach their own conclusions on whether your arguments pass muster.

Once you have a clear line on what you want to say, you would necessarily organise your data in a way that makes the point. It’s also often helpful to have a few paras of introductory text, talking about the visualisation, and the argument, since this sets the context in which users ‘read’ the viz.

If the target audience is a layperson, who has less domain knowledge of the relevant field, it’s more important to take them through the viz and the arguments you are making, especially if the field itself is complex. If the audience does have domain knowledge, you can certainly assume some familiarity of the subject.

How India Lives has taken an interest in disseminating data relating to socio-economic issues. What have been some of your personal experiences in working on public datasets with respect to the Indian context? Can you give some examples?

  1. Data is in forms and formats which make it difficult to parse (e.g. pdf, indeed we have seen a case where the data for download – from a government site – was an excel file, but which, when opened, contained only a scanned jpeg image of a data table. The site admin obviously had to fulfil a requirement that he disseminate the data in excel format, but had his own creative interpretation of what that entailed.
  2. Data is geographically incompatible. For instance, census data is based on districts as of census 2011. Since 2011, however the carving out of new districts, means that adding to that data is difficult without knowledge of how new districts map onto old ones. Further, the very concept of a ‘district’ differs depending on the public authority. For instance, police forces can have their own definitions of what constitutes a district, usually under the jurisdiction of an SP or DCP, and this is different from what the civil administration regards as a district. Thus, mapping crime data to other socio-economic data becomes a challenging exercise.
  3. Lack of clear GIS data. In India, there is no official, publicly available source of data, that is easily accessible, on GIS maps for the country, which remains updated to reflect latest geographical boundaries, both internal and external (e.g. has the government changed its maps to reflect the recent treaty with Bangladesh? If so, has this been released?).
  4. Data is in silos. Data released by one government department doesn’t necessarily map onto data released by another in geographic terms. (See point 2 above)

Despite this, our experience of working with public data has been hugely rewarding. The data can be complex, often confusing and maddeningly so. But taking the time to understand its complexity yields rich rewards in terms of understanding diverse socio-economic phenomena.

What is your take on visualising some of the new and non-traditional data inputs that are currently available? Also we are witnessing the movement towards a more “open data” architecture driven by the government, for instance through data.gov.in, that provides vast volumes of public data, what is your take on this?

Any tool, new or not, is only useful, when it is able to provide clear perspective on questions that the user/client is concerned with. Such tools include dashboards which allow the user to ‘cut’ the data in various ways, can be a very useful technique which allows the exploration of complex datasets. They also cover statistical techniques which, if used with knowledge of the underlying assumptions, can throw light on patterns within the data that are not immediately apparent.

As for the movement towards open data, this is a great move and data.gov.in certainly stands out among the range of government sites, in terms of ease of use. But individual departments and ministries should have a clear policy on releasing data at periodic intervals. Until this happens, the open data policy of the government will be implemented only partially.

With the increasing digitisation of public services, citizen level data trails are now being created and captured in government-created databases. What, if any, of these kinds of data do you think should be in the public sphere and what are the measures to be taken for data protection and privacy?

Data at the level of the individual citizens, such as names, mobile numbers etc, are obviously highly sensitive and should not be released, except under restricted circumstances (e.g. for researchers with the stipulation that they release data only in aggregated form). If released to the public, the data must be anonymised in a way that makes it difficult to trace the original identity of the citizen. Such data can also be released in more highly aggregated ways – e.g. at the level of a tehsil or district.


The Nexus of Financial Inclusion and Stability: Implications for Holistic Financial Policy-Making


Guest Post by Dr. Martin Melecky, Lead Economist, South Asia Region, World Bank

Both financial inclusion and financial stability are high on international policy makers’ agenda. For instance, the G-20 has called for global commitments to both advancing financial inclusion (the Maya Declaration and the Global Partnership for Financial Inclusion) and enhancing financial stability (the Financial Stability Board, Basel III Implementation, and other regulatory reforms). One challenge is that there can be important policy trade-offs between the two objectives.

A rapid increase in financial inclusion in credit, for example, can impair financial stability, because not everyone is creditworthy or can handle credit responsibly—as illustrated in the last decade by the subprime mortgage crisis in the United States and the Andhra Pradesh microfinance crisis in India. In addition, trade-offs between inclusion and stability could arise as an unintended consequence of bad or badly implemented polices.

At the same time, there may be important synergies between inclusion and stability. For example, a broader use of financial services could help financial institutions diversify risks and aid stability. Similarly, financial stability can enhance trust in financial systems and the use of financial services. It follows that understanding the synergies and trade-offs is paramount for policy makers who strive to advance financial inclusion and stability in tandem.

When evaluating financial sector outcomes, and prioritizing the design and implementation of alternative financial policies, policymakers could miss important aspects by ignoring the interactions between financial stability and inclusion. To illustrate this point, it is useful to consider the following intuitive framework:


Deploying policies to achieve financial stability and policies to achieve financial inclusion may not deliver the intended results if there are major tradeoffs between the two outcomes. But if the deployed policies can generate synergies between inclusion and stability, mutual reinforcement of the two goals can occur. The last term in the equation above highlights the possible interdependence between inclusion and stability, which can thus either add or subtract from the independent goals of stability and inclusion. While most studies and policies have typically focused on either achieving the outcome of stable or inclusive financial systems independently, limited attention has been paid to the interdependence between the two outcomes.

In a recent paper, Cihak, Mare, and Melecky examine a wide array of measures of household and firm inclusion to estimate an overall tradeoff between financial inclusion and stability. They find that, particularly for individuals, the use of financial services is negatively correlated with higher bank capitalization. Moreover, there is a positive correlation (tradeoff) between many inclusion indicators and the costs of banking crises. Greater financial inclusion (increase in account ownership or debit card penetration) is associated with more costly financial crises (output and fiscal costs, as well as the peak NPL ratios during crises).

Interestingly, synergies between inclusion and stability are almost equally probable as tradeoffs—as indicated by the two-peak (bimodal) histogram or correlations in Figure 1. Dissecting financial stability into resilience measures, volatility measures, and crises measures reveals that financial inclusion can help mitigate volatility of growth in bank deposits and the volatility of bank deposit rates. While financial inclusion of individuals, such as account ownership, use of electronic payments, formal savings and credit, help reduce the volatility of bank deposit growth and bank deposit rates, savings by firms can help enhance financial stability across all three dimension: resilience, volatility, and low probability and cost of crises.

Figure 1: Although tradeoffs between financial inclusion and stability prevail on average, synergies between the two outcomes could arise with almost equal probability


The relationship between inclusion and stability is systematically influenced by country characteristics, such as financial openness, tax rates, education, informality, population density, and the depth of credit information systems. While financial openness and formalization of the economy increases tradeoffs between inclusion and stability, low tax rates, education, and credit information depth help generate synergies between the two goals (Figure 2).

Figure 2: The inclusion-stability nexus is systematically influenced by country characteristics


Greater financial openness and movement of capital is particularly challenging in middle and low income countries, which tend to have a limited capacity to manage capital flows and ensure prudent and efficient allocation of the funding to creditworthy firms and individuals.

Countries with higher informality, as measured by the number of years firms operated without formal registration, experience a lower tradeoff between financial inclusion and stability. A potential explanation is that previously informal firms that enter the formal sector tend to be greater risk-takers. Being higher risk-takers may have allowed these firms to earn higher returns to pay for more expensive informal credit. Because risk appetites are unlikely to change fast after becoming formal, rapid increases in credit to previously informal firms that enter the formal sector should be monitored for potential threats to financial stability.

Low tax rates may generate synergies by stimulating precautionary savings due to smaller social safety nets and greater probability of unexpected increases in taxes. Education can generate a positive relationship between inclusion and stability by improving financial literacy and responsible financial inclusion that helps the financial system reap the benefits of economic scale and risk diversification.

The depth of credit information systems generates synergies by improving screening of creditworthy customers, including new users of credit, and aids stability by, for example, improving the accuracy of estimations of expected losses. Finally, greater information depth also promotes competition in oligopolistic markets, decreases the cost of finance, and encourages more firms and people to start using a financial service or use more than one financial service. Particularly if financial policy focuses on advancing the financial inclusion of individuals, complementary policies to deepen credit information systems could help mitigate the estimated tradeoffs with financial stability.

These findings have important policy implications. Because tradeoffs and synergies between financial inclusion and financial stability are significant, they need to be addressed in policymaking. In many countries, multiple government agencies (in many countries the central bank and other financial supervisors) and ministries (in many countries the ministry of finance, economic development, or strategic planning) are responsible for policy on both financial inclusion and financial stability. Therefore, the tradeoffs and synergies must be addressed at a high enough policy-making level to ensure effective coordination. One important tool to formulate high-level policy for the financial sector are the financial sector strategies that could be exploited for that purpose (Maimbo and Melecky, 2015).