Originally published in Financial Times, October 8, 2002

The Chinese Communist party is engaged in a historic struggle to salvage its soul. To observers in the west, focused on the politics of succession ahead of the party's November 8 congress, the idea that this 81-year-old monolith is in crisis may seem preposterous.

Yet there is a growing sense in Beijing that the party's greatest weakness is its inability to adapt to China's fast-changing society and economy. Bland official pronouncements from Beijing cannot conceal the fact that China's leaders are more worried about the health of the party than the seating arrangements in the politburo.

In particular, they see three trends that could lead to the loss not only of the Communist party's soul, but ultimately to its hold on power.

The party is suffering from an identity crisis: since the end of the Mao era, pragmatism has replaced ideology as the guiding principle for policy. The shift from leftist extremism to pragmatism after Mao Tse-tung's cultural revolution (1966-76) led to the reforms that have since transformed China. But although the country's leaders have abandoned the official communist ideology in practice, they continue to adhere to it in rhetoric and use it to justify the party's rule.

As a result, the discrepancy between official ideology and actual policy has grown so large that it is impossible for the regime to justify most of its policies on the basis of its professed values. This gulf has bred cynicism among the elite and alienated the masses; the party has lost its political identity and become little more than a coalition of self-interested elites.

The party's social base has narrowed sharply. Since the late 1970s, it has evolved into an elite bureaucratic grouping.

This transformation has brought in a new generation of technocrats that is now the mainstay of the regime. But at the same time there has been a huge erosion of the party's social base, mainly through the loss of support from agricultural and urban workers. Economic reform has degraded the party's organisational strength in these two groups: agricultural de- collectivisation dismantled the people's communes, while restructuring liquidated tens of thousands of state-owned enterprises.

As these two groups have little voice in the political process, party officials see little need to protect their interests. Government policies, such as high taxes in rural areas and industrial restructuring, have alienated them. The party is attempting to widen its support base, but its recent policy of admitting private entrepreneurs is not the way to do it.

Entrepreneurs are beneficiaries of the regime's policies and have access (through bribery) to government officials. Issuing them with membership cards will spawn crony capitalism, not help to expand popular support for the party.

The party is a dysfunctional organisation. With no external constraints, its health depends entirely on its internal mechanisms of accountability and incentive structure. To be sure, managing a behemoth of 66m members is a challenge beyond most leaders. But the party's embrace of market economics and rejection of democracy have made matters worse. Over the past 25 years, economic reform has greatly expanded the power of local officials. But as a hierarchical organisation, the party has failed to monitor its newly-empowered agents and hold them accountable.

Over time, decentralisation without accountability has created a collusive local officialdom whose interests are no longer aligned with those of the party. This collusive network is also insulated against public censure because China's closed political system denies ordinary citizens the power to enforce political accountability. Under this system, the penalties for misbehaviour are negligible and rewards for good performance almost non-existent. Corruption and irresponsiveness thus become institutionalised.

The late Deng Xiaoping foresaw the perils of a deteriorating Communist party two decades ago. "If China were to have a catastrophe," Deng warned, "it would first start inside the party."

Developments since then appear to confirm his prescience. Whether the party can reverse these dangerous trends - and salvage its political soul - hinges on its leaders' commitment to long-delayed political reforms. The main attraction at the 16th congress will not be the new leadership line-up, but the party's revised charter setting a new direction.