in internet traffic, threatened to make the work of GCHQ and NSA impossibly difficult. Soon the world was sending several million emails a second, and not even the great sigint leviathans could read them all. The days of the super-secret sigint agencies seemed numbered. However, in the 1990s Britain’s prominent role in the wars in Bosnia and then Kosovo reminded government that the need for sigint is perennial. In these Byzantine conflicts, the radio experts at Cheltenham were never quite sure which of the many different former Yugoslavian factions their various friends and allies were supporting.
Bitter conflicts such as Bosnia helped to convince Whitehall and Westminster that GCHQ was worth new investment. In 1996, under the direction of Sir David Omand, GCHQ began to develop plans for a remarkable new intelligence headquarters that quickly became known as ‘the Doughnut’ owing to its circular design. The intention was to bring all the staff together under one roof for the first time. Absorbing no less than fifteen miles of carpet and several hundred miles of fibre-optic cabling, ‘the Doughnut’ constituted the largest secret intelligence headquarters outside the United States. However, by the time it was completed in 2003, it was already too small. GCHQ had by then undergone a crash expansion following the 9/11 terrorist attacks. Its employees, now numbering more than 5,200, were soon ‘hot-desking’. By 2008, subsidiary buildings were being planned as the workforce approached 6,500.
Today, in somewhat cramped circumstances, GCHQ struggles with some of the most difficult issues of the twenty-first century. Not only is it the leading edge of Britain’s struggle against al Qaeda, it is also involved in fundamental issues of freedom and privacy that will shape the future of our society. Over the last decade, Britain has engaged with global e-commerce and finance more enthusiastically than perhaps any other country in the world. Our porous electronic borders present their own enormous problems. Globalisation, and in particular the global communications revolution, has brought many benefits, but it has also allowed miscreants to communicate and organise anonymously. The need for GCHQ to monitor both terrorists and organised crime means that the distinction between domestic and foreign communications has less meaning than it once had. GCHQ used to be a wholly outward-looking foreign intelligence service, but this is no longer the case.
Who will rule the internet? Will ordinary citizens be allowed genuinely confidential communication through encryption? How will artificial intelligence change intelligence in the middle of the twenty-first century? How much resource should we allocate to the new realm of cyberwar so we can hack back against Britain’s attackers? Or should we push for arms control in cyberspace? These are the questions that GCHQ ponders daily at the end of the second decade of the twenty-first century. Britain is already one of the most watched societies in the modern world and some would argue it is now addicted to surveillance. In 2008, Britain unveiled a £12 billion project to modernise interception. The following year GCHQ announced a remarkable project entitled ‘Mastering the Internet’ that collects the details of Britain’s communications and internet traffic for security purposes. In 2013, Edward Snowden revealed that GCHQ had done precisely that, at least in the sense that it had managed to keep up with the tsunami of digital information that was now hurtling around the world, mostly through fibre-optic cables. Even Britain’s Director of Public Prosecutions warned that things had gone too far.
Yet others argue that the surveillance society is largely something of our own devising. The most ubiquitous engines of surveillance are in fact Facebook and Google. Microsoft increasingly behaves like a state and certainly enjoys more revenue and political influence than many small countries. Private companies undoubtedly do worse things with our data than GCHQ. Meanwhile, faced with serious electronic attacks on the National Health Service and alarming evidence of electoral interference by Putin’s Russia, perhaps it is time to spend more on GCHQ, not less? Certainly, there are dangerous enemies determined to do us harm – including organised crime – that we can only be protected from with intelligence provided by GCHQ. Either way, because its role now combines leading on technological aspects of intelligence, and also securing the internet for the everyday citizen, GCHQ now finds itself at the centre of controversies that are of immense public importance. Accordingly, the time is now ripe to trace GCHQ’s long and secretive journey from the historic huts of Bletchley Park – via the Cold War – towards what now looks increasingly like a Brave New World.
‘How wonderful!’ I said. ‘Do you mean we’re overhearing Portsmouth ships trying to talk to each other – that we’re eavesdropping across half South England?’
‘Just that.’
Rudyard Kipling, ‘Wireless’, 1904[1]
In December 1902, Guglielmo Marconi made history by sending the first wireless radio message across the Atlantic. Remarkably, only two years later, Rudyard Kipling foretold the possibility of exploiting such radio messages to gather intelligence. In 1904 he published a short story entitled ‘Wireless’ that focused on intercepting communications sent from Morse equipment on board Royal Navy ships off the Isle of Wight. Kipling is thought of as a quintessentially late-Victorian author, but here he looks to the future, more in the manner of H.G. Wells, as his characters fret over technical matters such as induction and radio frequencies. To the readers of this fictional first instance of radio interception, the process seemed utterly magical. The Morse instrument ‘ticked furiously’, and one of the listening party observes that it reminds him of a séance, with ‘odds and ends of messages coming out of nowhere’. His companion retorts that spiritualists and mediums ‘are all impostors’, whereas these naval messages that they are eavesdropping on are the real thing.[2]
Kipling’s ‘Wireless’ is the first public discussion of the secret business of signals intelligence, or ‘sigint’. The magical process of extracting information from the ether would be one of the twentieth century’s most closely guarded secrets. Initially, producing ‘sigint’ only required equipment that would allow a third party to eavesdrop on a conversation broadcast by a radio transmitter using ‘wireless telegraphy’, but as this possibility became more widely known, communicators often resorted to using cyphers to keep their messages private. Thereafter, producing sigint usually required skilled listeners to capture the message and then a team of code-breakers to unscramble it. If the message was sent by cable rather than wireless, the listening-in process could be no less difficult than the code-breaking, or ‘decyphering’.
What did Britain’s code-breakers make of Kipling’s public airing of their black arts? The simple answer is that there were none to ask. Indeed, there had been no British code-breakers for more than fifty years. In the distant past, Britain had possessed a ‘black chamber’ in which skilled ‘cryptanalysts’ had broken the codes contained in diplomatic correspondence and private letters. These arcane skills resided in the ‘Secret Department’ of the Post Office. However, in 1847 this was exposed in a scandalous episode when the House of Commons heard that the Home Secretary had ordered the interception of the private correspondence of the heroic Italian nationalist in exile, Giuseppe Mazzini. Shocked Members of Parliament ordered an inquiry, leading to the closure of the ‘Secret Department’, just as the telegraph initiated what we now understand as a Victorian communications revolution. By 1904, Britain had been without a code-breaking centre for more than half a century.[3]
The immediate origins of MI5 and its sister service SIS (often known as MI6) can be traced to scares about German espionage in 1909. But British code-breaking was not revived until the very eve of the First World War. On 2 August 1914 the British Army set up a secret code-breaking section called MI1b. Soon, specialist Army units at various locations in Europe and the Middle East were busy intercepting German radio communications. One of the largest sites was the intercept station in Mesopotamia. In December 1916 the military code-breakers of MI1b were given a fabulous Christmas present when the drunken chief of the