We are often told that one of the great virtues of Indian democracy is its federal character. From the outset, India’s founding fathers eloquently argued that it was only through federalism that the nation-state of India could survive after independence. Decentralized power provides space for regional or linguistic identities to assert themselves. States—the constituent units of a diverse Indian union—are free, albeit with some notable constraints imposed by the center, to shape their economic agendas. Federalism allows states to exercise autonomy over critical areas of day-to-day governance. As defined by the Constitution, authority over many crucial sectors—from public health to land to law and order—lies with the states. In numerous other areas—education comes to mind—the states and the center share concurrent responsibility.

Yet federalism is heralded not simply for what its constituent parts are free to do on their own, but also because of how they are able to interact with one another. In the memorable words of the American Supreme Court justice Louis Brandeis, “It is one of the happy incidents of the federal system that a single courageous state may, if its citizens choose, serve as a laboratory.” The very existence of a federal union—in India or elsewhere—is thought to provide a venue for states to experiment, compete, and even learn from one another. Federalism, in theory anyway, creates a marketplace for public policy, in which the best policies eventually take hold and are replicated across units, while the worst are relegated to the dustbin of history.

As India’s economic prospects have dimmed and the center has been dogged by corruption scandals and policy missteps over the last 12 months, many inside (and outside) India have turned their attention from the center to the states—in hopes that a competitive spirit amongst the states will be India’s pathway to better economic and social outcomes, not to mention more robust democracy.

Against this backdrop of optimistic expectation, when surveying the present scene in the states, one is faced with a quandary: given the incentives for inter-state competition and rivalry, why aren’t India’s states learning more from one another, especially the laggards from the leaders? Put another way: for all the talk about India’s federal design serving as a laboratory in which state-led experiments can take place, why aren’t we seeing more successful policy experiments being piloted in one place and then replicated in another? This is not to say that there are no good examples of the diffusion of successful policies—mid-day meals in Tamil Nadu, rural employment guarantee in Maharashtra, the right-to-information acts in Delhi and elsewhere were all successful state experiments which later received nationwide acceptance. Yet in many of these instances, it was the center that picked up an idea and then legislated it from the top-down. Organic, bottom-up diffusion seems less common. Quite often, successful policy experiments at the state-level don’t spread laterally (from state to state) at all.

Consider the following thought experiment. One could argue—with the caveat that the media hype has somewhat outpaced the ground realities—that the Nitish Kumar government in Bihar from 2005 to 2010 delivered one of the most marked governance turnarounds we have witnessed at the state-level. Kumar successfully targeted three reforms with transformational potential: fixing the broken state police machinery and re-establishing the state’s monopoly on violence; refurbishing Bihar’s atrocious road network; and providing incentives for parents to send their daughters to school (and keep them there as they get older). So why haven’t more chief ministers of India’s traditionally backward states successfully cribbed from his first-term playbook?

The short answer is that there are institutional design features inherent in Indian democracy that conspire against a spirit of “competitive federalism.” These features are well entrenched, but not completely irreversible. For starters, the nature of India’s fiscal federalism is characterized by a severe vertical imbalance. This means that the states raise relatively little in the way of revenues, yet are saddled with the bulk of the expenditures. By design, states are beholden to generous transfers from Delhi in order to plug the financing gap. As a result of these skewed incentives, there is a still a tendency among some state leaders to believe that it is their job not to look around for ideas about best practices worth implementing, but to maximize the amount of monetary transfers they can procure from the central government. This exhibits the vestiges of an old tendency, whereby state leaders would pin their hopes on Delhi to bail them out and, if that failed, resort to playing the hapless victim “neglected” by the center. Ironically, while states lobby for greater transfers, many of them struggle mightily to fully utilize the funds that are already provided to them. For its part, the center has overseen a proliferation of centrally sponsored schemes and projects, each with its own rules, restrictions and strings attached. This likely weakens state-driven policy innovation.

Even if India’s fiscal federalism could be rewired, its states are not exactly paragons of liberal democracy, providing fertile ground for robust debate about ideas on policy reform. Across India, there is a worrying concentration of power in most state capitals in the hands of the chief minister. There is, admittedly, fierce electoral competition in the states, but once a government is elected, the CM reigns supreme, democracy be damned. The concentration of power in the chief ministerial post, which is often a direct reflection of the concentration of power within political parties themselves, is indicative of an atmosphere in which the CM’s word is gospel.

In theory, the state legislatures could serve as a venue for constructive dialogue on policy innovation—that is, if they met frequently enough. According to data collected by PRS India, under Mayawati’s tenure, the Uttar Pradesh assembly met on average a measly 22 days per year. Even in relatively well-functioning Delhi, its MLAs met for the same number of days over 2012. In Narendra Modi’s Gujarat, the assembly met for roughly 31 days a year between 2007 and 2012. When assemblies do meet, they are clearly not functioning as they should. Few bills are scrutinized by committees and, in fact, many bills are passed without any discussion at all.

This concentration of power makes it easy for political exigency to trump policy innovation in state capitals, with the former often a more pressing concern than good governance or growth. Hence, ruling parties are more concerned with consolidating power behind a powerful clique or leader, reflexively resisting everything the opposition supports or championed when it was in power, or currying favor with the center for greater funds, than with experimenting with genuine policy reform. It is no wonder, then, that the states have often fought tirelessly to obstruct further devolution to the local level, and constrain experimentation by their own districts, for fear of losing control. Political expediency is also linked to a reluctance to look beyond one’s borders for policy solutions—how could a BJP-governed state possibly learn lessons from its Congress-ruled neighbor? Such attitudes speak to a parochial mentality in state capitals, but they are also about politics—and they serve as a hindrance to learning both within and across states.

This is not to let the center off the hook for lack of learning across the states. The simple fact is that there does not appear to be a meaningful venue for an ideas-based exchange between the states. When the first Five-Year Plan was being drawn up in 1950, the government dreamed up the National Development Council (NDC), a body ostensibly created to allow states to comment on the Plans as well as to debate and discuss important social and economic policy issues of the day. Fast-forward six decades and no one believes the NDC is a well-functioning entity.

For starters, the body—which comprises the PM, various union ministers and the respective chief ministers—meets just one day a year. The most recent meeting, held in December 2012, was notable only for the fact that Tamil Nadu Chief Minister Jayalalithaa walked out in protest after she was only allotted 10 minutes for her remarks. To be fair, this was probably for the best, as the meetings usually are used as an opportunity for CMs to speechify and grandstand, with little in the way of dialogue that might open the door to policy diffusion or debate. Government alone is not to blame in this regard; civil society and the research community must also devote greater attention to cross-state comparative analysis of the impact of state-level experimentation. The shift toward randomized evaluation of social policies in India has produced valuable studies about what works and what doesn’t, but here, too, there is a tendency to focus within states rather than compare outcomes across them. Clearly not everything that works in Kerala will work in Jharkhand, but even when this is the case, we need to better understand why.

The nature of federal fiscal relations, the concentration of power in state capitals and the absence of a venue for cross-state exchange of ideas are all biased against a more competitive federalism in India. If India’s states are to truly serve as effective laboratories for improving public policy, they must be liberated from these institutional impediments. Institutions are notoriously sticky, yet there are a few signs of hope on the horizon. Notwithstanding incentives to the contrary, India’s states often do manage to spur policy experimentation—but it just is not always clear how these experiments add up. We have Chhattisgarh linking smart cards to the Public Distribution System, Andhra Pradesh evaluating the impact of contract teachers on primary education, Gujarat reforming electricity by linking higher user fees to guaranteed service provision—the list goes on. This demonstrates that state governments can take initiative and ministers are able to carve out space to experiment when the conditions are right. But the next step is much harder: fostering an environment of learning across states. One positive development in this regard is the Planning Commission’s recent proposal to streamline the number of centrally sponsored schemes while increasing the amount of flexibility that states have with respect to programming funds. According to the draft proposal, centrally sponsored schemes would set aside up to 20 percent of allocated funds (10 percent for flagship schemes) for “flexible spending” by the states, which would give them the space to encourage local experimentation.

Inadvertently, the recent reform to allow greater foreign direct investment (FDI) in multi-brand retail could also provide a learning opportunity for states. Although it was characterized as a defeat at the time, the center’s decision to allow states to decide whether to the implement the new FDI regulations will create, as it were, a “treatment” and “control” group. Those states standing on the sidelines waiting to see whether the adopters sink or swim will have a golden opportunity to learn from their peers. If this counts as a policy defeat, India may well need many more like it.

This article originally appeared in the Caravan.