1. Suddenly Coming Alive (2003-2009) My journey into domestic politics began in Iraq. Later I realised how many of the people I had worked with in different parts of the world wanted to make a similar journey. A man who had been a political adviser to a governor in Afghanistan, an officer who wrote on the Helmand tribes, a UN staffer specialising in the Sahel, and a conflict-resolution specialist from Myanmar all approached me for advice on how to become members of the British Parliament. But that was much later. At the time my journey felt more unusual. I had first entered government service in 1991 as an eighteen-year-old Scottish infantry officer on a short-service limited commission. I had been in Indonesia as a British diplomat for the fall of President Suharto in 1998; had played a part in the international interventions in Bosnia and Kosovo and Afghanistan; and had spent a year and a half walking, twenty to twenty-five miles a day, across Iran, Afghanistan, Pakistan, India and Nepal.

My early career was spent in a world which seemed to be becoming rapidly less violent, and less poor, and where it even seemed possible to ''make poverty history''. When the US and the UK invaded Iraq in 2003, I was appointed to an Iraqi province, where I operated at first as the acting governor. Our money was apparently limitless - in my case delivered in vacuum-packed bricks of a million dollars in bills, which I could spend without audit. When things got sticky, 100,000 troops and AC-130 Spectre gunships stood by to back us up. I had entered Iraq supporting the war on the grounds that we could at least produce a better society than Saddam Hussein''s. It was one of the greatest mistakes in my life. We attempted to impose programmes made up by Washington think tanks, and reheated in air-conditioned palaces in Baghdad - a new taxation system modelled on Hong Kong; a system of ministers borrowed from Singapore; and free ports, modelled on Dubai. But we did it ultimately at the point of a gun, and our resources, our abstract jargon and optimistic platitudes could not conceal how much Iraqis resented us, how much we were failing, and how humiliating and degrading our work had become.

Our mission was a grotesque satire of every liberal aspiration for peace, growth and democracy. Most striking was not the failure, but the failure to acknowledge our failure. Politicians, ''experts'' on Iraq and counter-insurgency and many liberal advocates for state-building, continued to insist it was working - or if it wasn''t, this was simply because it wasn''t being ''done right'': that some new team with a new strategy could make it alright. The hysterical optimism at the highest levels was shadowed by the most profound cynicism on the ground. Too often, I and my colleagues, whether civilian or military, were encouraged to shy away from precise and honest descriptions of our failure, and instead to perpetuate worthless and extravagant projects designed to placate the imagined tastes of our political masters. I lived these paradoxes as a relatively senior official in the occupation government. On the one hand, I felt that someone like me should never have been governing an Iraqi province. On the other, I was completely immersed in the work.

During the days, I chaired meetings with senior Iraqi officials, visited schools we had funded, and tried to reason with crowds who sometimes waved banners calling for my death. In the evening I retired to my shipping container, which was wrapped, since a mortar had come through the roof without detonating, in a tea-cosy of sandbags. The trays in the dining hall and the shower blocks seemed a natural extension of the life I had known since my father - who had himself been a colonial administrator and British diplomat - sent me aged eight from our home in Malaysia to a boarding school in Oxford. But the many moments of individual courage and achievement, which I witnessed in a place of bombs and death and power far from home, were components of an illegitimate occupation. It took two years before my bewilderment at these failures and hypocrisies, and my part in them, drove me finally to resign from government service. I did not return to Britain. Instead, having been part of an attempt to make Iraq more like the United States, I decided to try to preserve what was unique about Afghanistan. I swapped the shipping containers and airbases in Iraq for a mud fort in Kabul, and set up a small charity on behalf of the Prince of Wales, who had developed a deep love of Afghan calligraphy and woodwork.

We worked to rebuild some of the houses of the old city of Kabul and support traditional craftsmen and women. The environment in central Kabul was worse than anything I had seen in Iraq. The old city had not been rebuilt after the bombardments a decade earlier. Collapsed buildings lay eight feet deep in the street and the lanes ran with oil-thick sewage. Recently returned refugees, possessing little except a few tin pots, huddled behind curtains of rough blankets, strung across the gaps in the mud walls. There was no clinic or primary school, adult life expectancy was thirty-seven, one in five children were dying before the age of five, and almost everyone was unemployed. Yet, stooping beneath the cracking lintels, and following the worn staircases, I found rooms decorated in ancient spirals of limestone plaster, set with glimmering glass. The inner facades of the buildings were panelled with blackened screens of cedar wood, carved into roses and lilies.

Abdul Hadi who was selling bananas in the street was a master of all the forms of Kabuli carving, and a former cabinetmaker to the king. Tamim, a miniaturist - who had been tortured by the Taliban, when they found his drawings - was trying to offer private art lessons. I started with a loan of £40,000 and one employee - my driver, whom I called my ''logistics manager''. My second employee - now called ''chief engineer'' - found a hundred spades and wheelbarrows and within a day we had employed all the unemployed men in the community, clearing garbage. Within a month, they had dropped the street level by six feet. Within six weeks, we were running craft lessons for women and men. The team went on to restore over a hundred buildings, constructing along the way an art institute, a clinic and a primary school. Working in big government jobs - governing an Iraqi province of a million people, for example - had not begun to prepare me for a start-up and running an NGO.

The Prince of Wales was engaged and immensely helpful. But I got a lot wrong. The clinic that I had resisted creating (''it is not in our strategic plan'') became the most successful part of the entire project, seeing 27,000 patients a year. I almost ran out of money twice. But it was the most fulfilling work I had ever done. I liked working with Afghans. The work was the antithesis of the Iraqi occupation and its utopian dreams. We worked quickly.

The young foreign volunteers who came to help on the project made me fall in love with Britain and the US again. I admired how hard they worked on their Dari, and in some cases on Pushtu and Arabic as well; how they put up with hand grenades and bombs; how they walked, and sometimes skied, in remote rural areas; collected books and plants. They were practical, effective and funny, with an ironic sense of their own limitations, honest about their lack of expertise, and sensitive and respectful to Afghans. First among them was Shoshana, formerly a middle-school science teacher in the poorest areas of New York and Boston, who became my deputy and, much later, my wife. Without this partnership between Afghans and foreigners the community would have left; Abdul Hadi would have died without passing on his woodworking skills to a new generation, and all those buildings would have been demolished and cleared for the mayor''s East German-style plan. So, the project made me optimistic about Afghans, and through the lens of the volunteers, about the West, and even about myself. Afghanistan itself, four years after 9/11 and my walk across the country, seemed transformed. In the highlands, where on my 2001-2 walk I had seen village after village burnt to the ground by the Taliban, I now found clinics and schools.

More than a million girls were going to school for the first time. Mobile phones seemed suddenly everywhere. Health and life expectancy were far better. Millions of Afghan refugees were choosing to return home. All this seemed to be a much better trajectory than Iraq, and I credited it to foreigners staying out, and keeping only a very light military footprint in Afghanistan. Except, my former government colleagues were reaching the opposite conclusion. They told me that Afghanistan was a corrupt, violent, drug-riddled catastrophe, which only they could save. A new generation of American heroes was posted to Kabul to fulfill this dream - generals who got up at 4 a.

m. to sprint eight miles around their airbase. They were not simply trying to pick up garbage in the old city. They were parachuting in, like turnaround CEOs, to fix the whole country. The immense confidence in US and UK power to transform Afghanistan was apparently unaffected by how difficult it had been to do anything in Iraq. Perhaps if they had been seeking to turn around lives in an ex-coal town in Durham or to work with Native American tribes in South Dakota, they would have paid more attention to the history of local communities, and been more modest about their position as outsiders. They might have understood that messiness was inevitable, and patience and humility essential. But somehow in Afghanistan - a place far more traumatised, impoverished and damaged than the ver.