Writing authoritatively about a spy service is hard for an outsider but Britain's is a particularly tough case.
Fact must be sifted from a big body of popular fiction, much by novelists with an intelligence background including James Bond author Ian Fleming and the current Hollywood version of John le Carre's "Tinker, Tailor, Soldier, Spy" starring Gary Oldman as spymaster George Smiley.
Gordon Corera, author of a history of the Secret Intelligence Service (SIS) from the Cold war to the present day, set out to solve this conundrum by persuading several former UK intelligence officers to tell him some of their best stories.
These personal recollections are blended with anecdotes culled from more narrowly focused histories and memoirs written by men and women of various nationalities who dealt with SIS, also known as MI6, while toiling in diplomacy or the armed services over the decades.
The result is "The Art of Betrayal: Life and Death in the British Secret Service" published by Weidenfeld and Nicholson.
He spoke to Reuters about the book.
Q: MI6 is one of the most written about organisations. It's a crowded field. What made you want to tell your version?
A: I wanted to tell the human story, in large part as that goes to the heart of what lies behind MI6's work in recruiting agents. I didn't want to write a history of committees, saying there were 13 desks looking at the Soviet Union on such and such a date and then x desks a few years later. I was looking for personal stories and motivations behind people like Oleg Penkovsky. That's what the service's work is about, what actually lies behind the act of spying. There's quite a lot in the public domain if you know where to look. So part of what I was trying to do was to bring together all that material but also add what I could from some particularly strong access to people like the late (SIS officer) Daphne Park. I had done a radio series where I had been into MI6 and interviewed the then chief John Scarlett. So I could tell an overarching story through to the present in a way that no one had done before.
Q: To what extent did you want to pick apart the magical reality that MI6 occupies in the public mind?
A: Fiction defines what we understand about British intelligence although many of the great fiction writers, whether it's le Carre, Ian Fleming or Graham Greene, had backgrounds in real intelligence work. I think that's a sign of how fact and fiction have become intertwined in a way which has become quite hard to separate, even for some of those of those within the organisation. What I was trying to do was to say here was the fictional understanding of MI6, let's see what it's really like -- James Bond, John le Carre.
The answer is that at certain times it is a bit like that. There are periods of bravado or aggression that have not ended very well. There have been periods of le Carre-like introspection which have also been quite difficult. Fiction offers an interesting way of understanding some of the cultures within the organisation, but equally I did want to say it isn't like the fiction. The truth is it's not about a licence to kill or some of the other myths.
But there has been an interesting debate within MI6 about how far the fiction helps to serve their purpose in recruiting officers and agents in the field? Some have long believed it's quite useful to have this mystique. But there there are others who argue - and who I think now have the upper hand - that it's actually a bit dangerous to over rely on the fiction. In the era of accountability and transparency it's not enough to rely on the fiction for people to have a better understanding.
Q: You described a service that was institutionally insecure about the way its bosses see it. Is that now resolved?
A: The insecurity in the past came out of secrecy. Precisely because it didn't officially exist, it didn't have a sound footing and could have been abolished at a whim. The insecurity was born out of some of its failures like the (Soviet agent) Kim Philby disaster. At the end of the Cold War, it does get a sound legal footing. Then you have a new crisis. There's a real sense of concern in the early 1990s as an organisation about what it is for. You've got people going up to the Chief in Whitehall and saying jokingly "Oh, your still here! We didn't realise you're still around." September 11 seemed to answer some of those insecurities but in doing so it drew MI6 into politics and into the firmament of Iraq and very hot issues like the relationship with America and how it treated its detainees. Few would dispute now that we need an intelligence service, but at the same by moving out of the shadows and into the light they're also drawing a lot more scrutiny and coming under a lot more pressure than they did before. I don't think that's a threat to MI6's existence but it's posed a new set of questions about how close it gets to policy and politics.
Q: Has the shadow of the Iraq weapons of mass destruction been removed from SIS?
A: Iraqi WMD was a shattering blow. The pride of the organisation is on recruiting good agents and here was one time when that was put on public display and it turned out to fall apart. Most of the sources just evaporated under scrutiny and that goes to the heart of what they are supposed to be doing which is recruiting agents and human intelligence. The only thing that's helped them is that there have been other issues since then for them to focus on and in particular dealing with the terrorist threat has given them a clear purpose.
Q: Do you get the sense that, if the files were opened or you had years more to research, you'd get a lot more stories, or do you feel you've got the best here, and the rest is a bit dull.
A: I think you'd find a few more triumphs and probably a few more disasters we didn't know about. I think there are a few agents run in the Cold War that have not come to light. It's easy for intelligence agencies to say all our successes are secret, our failures public. I'm not sure that's always true. (Edited by Paul Casciato)