Later this month, the World Bank will release the 2018 edition of its annual “Doing Business” indicators. This initiative attempts to quantitatively capture the regulation that small- and medium-sized firms encounter in 190 countries around the world. Established in 2002, the annual exercise has arguably become the single most influential measure of a country’s investment climate. One need look no further than the regular complaints filed by countries unhappy with their place on the league tables to understand that the World Bank’s metric has outsized influence.
One country where the Doing Business rankings have captured widespread attention is India, in part because it has generally fared quite poorly. Clocking in at 130th on last year’s rankings, India had the worst business environment of the BRICS (Brazil, Russia, India, China and South Africa) economies.
Even by Indian standards, the anticipation for this year’s edition of the report is remarkable—especially in government quarters—since there is an expectation that India’s rating will improve significantly thanks to recent reforms. Prime Minister Narendra Modi has staked a fair amount of personal credibility on India’s ratings success—frequently reiterating his desire to see India break into the top 50 countries in the world—and, with his first term drawing to a close in 2019, the sense of an impending deadline is palpable.
Whatever the outcome of this year’s report, observers should treat the rankings with care. Politicians and government officials should not be too quick to claim credit or point fingers as the case may be, nor should voters view the ranking as the ultimate marker of the ruling party’s reform success. Likewise, investors who are considering the prospects for investment in India should recognize what the rankings do and do not tell us.
Doing Business: De facto vs de jure
By the World Bank’s own admission, the Doing Business rankings do not “measure all aspects of the business environment that matter to firms or investors”. Rather, they provide an assessment of red tape and administrative hurdles across 11 areas of business regulation. To determine their rankings, the World Bank relies on four sources of information: the laws and regulations on the books, experts well-versed in local business practices, national governments, and World Bank staff. The World Bank uses these various inputs to estimate how many hoops a “prototypical firm” must jump through to carry out a set of standardized tasks.
The insights the World Bank compiles are extremely useful, but they are not necessarily representative of what firms experience in real life. As economists Mary Hallward-Driemeier and Lant Pritchett have noted, where the informal economy thrives and regulations are poorly enforced, the de jure regulations measured by the World Bank are often only tangentially related to the de facto processes that businesses encounter.
The challenges are even more profound in large countries such as India, since the World Bank has traditionally assessed just the largest business city in each country, only incorporating a second major city for the 11 most populous economies as of the 2015 report. In India, the relevant cities are Mumbai and Delhi. While this limitation will likely be addressed in the future— with the proposed inclusion of eight additional cities in the Indian case—for now, one should consider the representativeness of the data with caution and bear in mind that the methodology incentivizes reforms in a few cities rather than improvements to the investment climate of the country as a whole.
The World Bank, to its credit, has tried to account for these limitations through another undertaking: firm-level enterprise surveys. These are explicitly designed to capture the de facto realities that the Doing Business indicators might miss, shedding light on what firms actually experience, as opposed to what experts estimate or formal rules demand. However, enterprise surveys also have their shortcomings—even beyond the relative infrequency with which they are conducted—including recall bias and issues with sample size.
Comparing surveys
The World Bank’s most recent enterprise survey for India was carried out in 2013-14 but the IDFC Institute in Mumbai conducted its own survey (in 2015-16) of manufacturing firms in conjunction with NITI Aayog, the results of which were published last month. For comparison, we use the World Bank’s 2015 and 2017 Doing Business reports, which report assessments from 2014 and 2016, respectively. Leveraging these four sources of data—two years each of Doing Business indicators and enterprise surveys—one can gauge differences between the two methodologies as well as relative change over time. While the topics across different surveys are not perfectly aligned, two core questions were roughly the same: the number of days taken for a firm to obtain a construction permit and the number of days needed to get an electricity connection. Further, by limiting the data from the World Bank enterprise survey to the manufacturing sector, we were able to create largely comparable samples.
Figure 1 displays the number of days taken to obtain a construction permit. There are two obvious trends in this graph. First, the Doing Business indicators suggest that the waiting time to obtain a construction permit is considerably longer than what firms report in the enterprise surveys. According to the Doing Business data, in both 2014 and 2016, a typical firm should expect to wait an average of 190 days to get a construction permit in Delhi and Mumbai. Second, while the enterprise data report significantly less delay in obtaining construction licences, the waiting time appears to have grown over the past three years (from 31 to 52 days). On its face, this seems like a negative development. In practice, the implications are not that clear-cut: greater reported delays could be the result of increased enforcement or the elimination of a prominent channel of rent-seeking; factors that might not be readily apparent in the Doing Business reports. Alternatively, the differences in the data could be artefacts of the methodologies employed. For instance, the enterprise surveys include much broader geographic samples than the Doing Business reports, which could be an issue if regulation in big cities differs from that in smaller towns.
Figure 2 shows the number of days a firm must wait to get an electricity connection. In 2014, according to the Doing Business indicators, firms that follow all legal regulations could expect to wait an average of 102 days to get an electricity connection. Two years later, that number had dropped by more than half. In 2014, the wait time was significantly longer, according to the Doing Business indicators, compared to the firm-level enterprise survey (47 days longer, to be precise). By 2016, however, the two measures converged at roughly 50 days.
These comparisons are instructive, but not necessarily suggestive. To begin, we do not have access to the raw data for the IDFC Institute survey, which limits the analyses we can undertake. Additionally, the experiential measures used by the enterprise surveys rely on relatively small sample sizes since most queried firms did not report seeking construction permits or electricity connections in recent years (with numbers in the hundreds, despite samples of several thousands).
Implications
When the Doing Business report comes out this month, the nuances inherent in the data will likely be neglected by commentators looking to score points for one side or the other. Calmer heads should keep the following points in mind.
First, the Doing Business indicators provide a snapshot of a country’s red tape; they have no pretension of providing a comprehensive picture of the investment climate. As the World Bank makes clear, the indicators are not designed to comment on macroeconomic indicators or prospects for growth. Thus, while India may find itself sandwiched between Cabo Verde and Cambodia in last year’s rankings, it would be a mistake to conclude that they are comparable investment destinations.
Second, there exists a wide divergence between de jure and de facto realities in most economies. What firms actually encounter “on the ground” is perhaps more important, but there are limitations to our ability to measure and interpret those experiences without bias. The Doing Business reports’ de jure indicators offer a snapshot of a country’s regulatory cholesterol, but likewise should not be viewed in isolation. Rather, by using the two types of data in tandem, one can develop a more holistic picture of the business environment. Furthermore, by examining the differences between the data sets, one can gain insights into issues of governance and the rule of law.
Finally, one aspect of the Doing Business report worth watching is how India rates on the so-called “Distance to Frontier” (DTF) measures, which capture the ease of doing business compared to the highest score any country has ever received in a given category (say, registering property). This metric is useful because a country can make absolute progress but fail to climb in the relative rankings because other countries have also reformed. Consider a runner who is placed sixth in a race. That position means very little without additional information on the calibre of competition and some perspective on how close together the competitors were. Losing by two seconds in the 100m dash holds a very different meaning than a two-second defeat in a marathon. In sports as in life, context is everything.
This article was originally published in Livemint and Ideas for India.