Source: Ideas and Institutions Issue #5
Analysis
Measuring Performance in Government: the Example of National Time Release Study
On April 11, the chairman of the Central Board of Indirect Taxes and Customs released the National Time Release Study (NTRS), 2022. The study covers 15 customs formations under four port categories—seaports, air cargo complexes, inland container depots, and integrated check posts—that handle about 80 percent of the bills of entry (imports) and 70 percent of the shipping documents (exports). It measures the performance of trade facilitation in terms of the average cargo release time, which is the time taken from the arrival of the cargo at the customs station to its eventual release for import or export. Good performance on this is crucial for reducing frictions in the trading process.
The NTRS reports a significant reduction in the average import release time in 2022 from 2021 in all the four port categories. But it reports a significant increase in the average export release time at seaports, inland container depots, and air cargo complexes. In exports, the study noted that the documentary clearance of export consignments, which is the time taken to grant approval to move goods out of India, once they have arrived, has been significantly reduced. However, the time taken to export goods, post-clearance, has increased. This may be due to frictions in global supply chains, infrastructural issues at the ports, or uncertainties associated with transport connectivity between hinterland and ports.
While the findings are noteworthy, the study is also an interesting example of how the government can systematically develop and scale performance measurement systems.
Trade facilitation has been an important issue for long, but in recent years, the impetus has come from the World Trade Organization’s (WTO) Trade Facilitation Agreement, which came into force in February 2017. The agreement includes measures to expedite the movement, release, and clearance of goods. A National Committee on Trade Facilitation, chaired by the cabinet secretary, makes National Trade Facilitation Action Plans (NTFAP) to ensure that the agreement is implemented. The first plan covered the period between 2017 and 2020, and the second—from 2020 to 2023—is ongoing. Among other things, the NTFAP sets targets for the release time of imports and exports for the four port categories.
The NTFAP refers to four benchmarks for the government’s efforts: the Organisation for Economic Co-operation and Development’s (OECD) trade facilitation indicators; the World Bank’s trading across borders, a component of the World Bank’s EODB Index; the World Bank’s logistics performance index; and the World Economic Forum’s global competitive index.
These international measures can be useful for comparing performance, but each country should also develop its own measures to verify their validity, reliability, and objectivity. Methodological problems, especially the subjectivity in assessment, the focus on de jure requirements or perceptions rather than de facto performance, and limitations of sampling often plague international measures. Such issues led to the discontinuation of the World Bank’s EoDB Index.
The NTRS builds on years of effort to measure the performance of trade facilitation. Time release studies were first conducted and iteratively improved in specific customs formations. These studies informed the national implementation and policy reforms. For instance, the Jawaharlal Nehru Customs House (JNCH), Nhava Sheva, regularly conducted the studies 2012 onwards, but they initially included only imports and excluded cases where advance filing of documents had been done. In 2018, they included both exports and imports and included cases of advance filing of the documents, the latter being necessary to capture the benefits of pre-arrival processing of documents. This methodology was stabilized and tested. All these studies were conducted in-house by the customs officials.
In 2019, an NTRS was done on a trial basis. Then, in 2021, the methodology was finalized, broadly based on the learnings from the JNCH studies and the 2019 study. The national study was then conducted and published. While the earlier studies in specific customs formations were conducted by customs officials, an external consulting firm was engaged for the national study. It is worth noting that since the process flow is reported online, it helps track the movement of each bill of entry and shipping document.
Such iterative development of methodology is worth considering as a practice in other public administration contexts. Iterative development of measures, by testing them in real world circumstances, before fully integrating them in the administrative functioning can help improve their quality.
The development of time release studies also fed into the policy process, and some of the officials took the initiative to conduct research using these datasets. In matters of public administration, reflective analysis by serving officials can often be useful because they understand the constraints and incentives well.
In their paper, Vijay Singh Chauhan and Arun Prasath K. identified three sets of issues leading to delays: the time importers take to submit import documents; the level of facilitation or intervention, as determined by the risk management system; and the involvement of non-fiscal concerns—primarily related to regulations in domains such as drugs and food safety, among others, that need to be processed by authorities other than customs. Such problem analysis informed policy responses. For instance, the concern around the timing of submission led to amendments of the law to increase advance filing of documents, which helped expedite the process.
Another paper, by Vijay Singh Chauhan and Sruti Vijaykumar, focused on using these studies to evaluate the international measures. They considered the four international benchmarks discussed earlier and compared their methodologies and findings with those of the time release studies. They argued that the technology-enabled, data-driven time release studies are a better metric for measurement of performance than the survey or perception-based indicators. Interestingly, they found that India’s scores for 2017 and 2019 on the OECD’s Trade Facilitation Indicators were revised long after they were initially released, leading to a considerable improvement in the scores. It seems that this revision was done in response to the time release studies published in India. So, if robust measurement systems are developed by a country, they can help improve the quality of the international measures as well.
Coming back to the time release study, is such a study useful for administrative purposes? I have witnessed many debates around measurement in education, health outcomes, regulation, and other areas, where a lot was said without properly engaging with the practical challenges of performance management in public administration. While we should not pit the perfect as the mortal enemy of the good, it is important to read these studies carefully.
First, we should consider that once a measure becomes a target, it ceases to be a good measure (an oft-quoted version of Goodhart’s Law). However, as a measure gets close to capturing the overall performance of a system, and not a particular aspect, it becomes increasingly difficult to game. Release time is a consequence of many factors, all of which cannot be gamed. But the results should indeed be seen as achieved on a best-efforts basis. This Hawthorne effect is to be expected because, to avoid seasonal effects, the data is collected during a particular period each year. Since the customs officials know this, they must be putting their best foot forward during that time. The study is still useful for year-on-year comparison of the best-efforts performance.
Second, this measure captures only the output (e.g., release time) and not the outcome (e.g., public value created by trade facilitation). For the purpose of holding the customs administration accountable, the output is useful. The public value, which is difficult to measure, is an outcome of several outputs. Customs officials should be held accountable for things that they are, more or less, in a position to influence.
Third, it is important to see the factors external to trade facilitation that may have influenced the results for better or for worse. This year, for instance, the frictions in supply chains seem to have influenced performance on exports. Further, since authorities other than customs administration also participate in the process, the overall performance may be due to delays by other authorities. The study could be used to improve accountability of other regulatory agencies and to nudge private custodians. Fourth, while it is better to have faster release of imports and exports, excessive focus on this one metric may be counterproductive. Since the customs and other authorities involved in the approvals have regulatory functions, it is important that they verify, albeit based on a pragmatic assessment of risks, before giving approvals. So, attention to capacity improvements must accompany the ambition to improve release time. An argument one sometimes hears is that measurement of performance in government is counterproductive, as it places emphasis on measurement rather than performance. Since measurement is always partial, focusing on what gets measured can crowd out a thicker understanding of public service. However, if done well, this tension between improving professional ethos among civil servants and making them accountable can be minimized. High quality measurement of outputs (and if possible, outcomes) can allow the government to give more flexibility to civil servants to exercise their professional competence in solving the problems. In the case of trade facilitation, this takes the form of developing systems to focus on the high-risk cases while taking a more trust-based approach with others. The development of NTRS seems to be a good example of building and using a measurement system based on real world experience. Other domains of public administration could learn from it.
—By Suyash Rai
Review
Due Process and Article 21
What is the right balance between substantive due process rights and the imperative of economic development? This question continues to be relevant today but was especially significant while India’s constitution was being drafted. Rohan J. Alva’s book Liberty After Freedom: A History of Article 21, Due Process and the Constitution of India provides a detailed account of the considerations and conflicts the drafters of the Constitution faced while deciding whether to incorporate an expansive or restrictive version of the right to life and liberty under Article 21.
As Alva rightly notes, Article 21 has become one of the key provisions of the Constitution under which India’s higher judiciary has expanded the nature of fundamental rights. He notes that "the Supreme Court has found a home for over thirty fundamental rights...from those who live behind prison walls to those who toil to earn a living in the most trying circumstances." In recent public memory, the Supreme Court has decided that a right to privacy exists under Article 21, by interpreting the right to life expansively.
Alva’s book shows that the drafters of the Constitution (chiefly, the Drafting Committee of the Constituent Assembly and Sir B.N. Rau, constitutional adviser), were acutely aware of the fact that a right to due process, under the Constitution, could possibly be read expansively. The key debate was whether to provide (a) a limited guarantee that life and liberty could only be constrained by a procedure established in law, i.e., procedural due process, or b, to incorporate a provision that would be more likely to be read expansively—a substantive due process right.
He traces the development of due process jurisprudence in the United States and highlights how the drafters of the Indian Constitution were acutely aware of how this jurisprudence was, at that time, stymieing progressive legislation. The first discussion in the Constituent Assembly culminated in an aspiration to adopt an expansive due process clause. Sir BN Rau—one of the key individuals involved in the Constitution’s drafting—studied the development of jurisprudence in the United States as well as similar constitutional provisions in Japan and Ireland and proposed a middle path:
- The right to liberty under Article 21 should be suffixed with the word personal, so that courts would interpret liberty as individual liberty, rather than economic liberty.
- The right to property should not be subject to due process rights, as this would, in all likelihood, interfere with the then desirable objectives of undertaking extensive land reform in the country. Rau and some other drafters were worried about land reform being stymied by a surfeit of litigation under Article 21.
- At the same time, the Irish model, where life and liberty could be taken away by a validly enacted law, was not adequate. In his book, Alva talks about how Rau rejected the model of the Irish Constitution out of the fear that “once a fundamental right dealing with personal liberty was entirely dependent upon the law-making powers of the State, then all that an oppressive state [would] need to do is pass a law which restricts the fundamental right.”
Therefore, according to BN Rau, a middle path was desirable, where due process rights were provided only to protect personal liberty.
However, Alva shows, even this formulation was eventually curtailed. Due process was almost shelved by the Drafting Committee of the Constituent Assembly. Alva argues that Mahatma Gandhi’s assassination and the ongoing violence due to India’s partition motivated the drafters of the Constitution to consider the imperative of using state power to maintain law and order. Alva objects to this strongly. He says, “ultimately, the formulation of the right to life and personal liberty, without due process, was an act of impulsive spontaneity, unsupported by strong reasons justifying a departure from the due process guarantee.”
Alva argues that India eventually retained some semblance of due process guarantees only because of the strident opposition of other members of the Constituent Assembly and objections raised by civil society. In response to both, the Drafting Committee introduced a new provision (Article 22) providing protections against arbitrary arrests and detention, but which also permitted laws allowing preventive detention.
It is clear, from the account given by Alva, that the drafters and the Constituent Assembly deliberately rejected a due process right similar to the U.S. constitution. Article 21 today provides that life and liberty can be taken away as per procedure established by law. The fact that the Indian Supreme Court has interpreted this limited procedural guarantee expansively, is, according to Alva, an unalloyed benefit. Alva points out that “in the vast array of cases in which unenumerated rights have been read into Article 21, the motivating purpose for the Supreme Court...has been to advance social progress and public welfare.”
He also argues that this expansive interpretation is the correct way to interpret Article 21, given that there were procedural irregularities, volte faces, and strident challenges to the restrictive nature of Article 21 during the drafting of the Constitution. In Alva’s words, “Seen in this light and given this history of the Drafting Committee’s deliberations generally, and on Article 21 specifically, it is in fact fully possible to say that there is no original intention....which can be attributed to Article 21.”
As Alva’s detailed historical exposition highlights, the tension between an expansive notion of human rights and democratic imperatives is an enduring one. The Constitution was adopted when India was mired in poverty and illiteracy, suffering from large-scale violence and fissiparous tendencies. Today, India is a stable democracy but remains a third-world economy. In this context, where expansive constitutional interpretations can still interfere with democratic imperatives, it is difficult to agree with Alva’s claim that an expansive interpretation of Article 21 is an unalloyed good.
The Indian judiciary has used Article 21 to improve procedural fairness in the criminal justice system. In doing so, it has incorporated substantive ideas of natural justice that relate closely to ideas of personal liberty and procedure established by law in Article 21. However, the Indian judiciary has also read more than two dozen unenumerated rights into the same provision, many of which interfere or conflict with democratic imperatives. When the right to shelter, for example, is declared a fundamental right, it constrains democratic decision making on how important the provision of shelter is compared to other difficult choices governments must make. At the very least, the question of whether this judicial activism to expand the scope of Article 21 is good is debatable.
—By Anirudh Burman