Yalnız Litres-də oxuyun

Kitab fayl olaraq yüklənə bilməz, yalnız mobil tətbiq və ya onlayn olaraq veb saytımızda oxuna bilər.

Kitabı oxu: «GCHQ»

Şrift:

GCHQ
Richard J. Aldrich

The Uncensored Story of Britain’s Most Secret Intelligence Agency


Copyright

HarperPress

An imprint of HarperCollinsPublishers Ltd.

1 London Bridge Street

London SE1 9GF

www.harpercollins.co.uk

Published by HarperPress in 2010

Copyright © Richard J. Aldrich 2010

Richard J. Aldrich asserts the moral right to be identified as the author of this work

A catalogue record for this book is available from the British Library

All rights reserved under International and Pan-American Copyright Conventions. By payment of the required fees, you have been granted the non-exclusive, non-transferable right to access and read the text of this ebook on screen. No part of this text may be reproduced, transmitted, downloaded, decompiled, reverse engineered, or stored in or introduced into any information storage and retrieval system, in any form or by any means, whether electronic or mechanical, now known or hereinafter invented, without the express written permission of HarperCollins ebooks.

HarperCollinsPublishers has made every reasonable effort to ensure that any picture content and written content in this ebook has been included or removed in accordance with the contractual and technological constraints in operation at the time of publication

Source ISBN: 9780007312665

Ebook Edition © JUNE 2010 ISBN: 9780007357123

Version: 2016-08-23

For Libby (for the dark night-time)

Table of Contents

Cover Page

Title Page

Copyright

Sigint and Comsec Locations in the UK

Note on Terminology

Abbreviations

Introduction

THE 1940s BLETCHLEY PARK AND BEYOND

1 Schooldays

2 Friends and Allies

3 Every War Must Have an End

4 The KGB and the Venona Project

5 UKUSA – Creating the Global Sigint Alliance

THE 1950s FIGHTING THE ELECTRONIC WAR

6 ‘Elint’ and the Soviet Nuclear Target

7 The Voyages of HMS Turpin

8 Sigint in the Sun – GCHQ’s Overseas Empire

9 Blake, Bugs and the Berlin Tunnel

10 Embassy Wars

THE 1960s SPACE, SPY SHIPS AND SCANDALS

11 Harold Macmillan – Shootdowns, Cyphers and Spending

12 Harold Wilson – Security Scandals and Spy Revelations

13 Intelligence for Doomsday

14 Staying Ahead – Sigint Ships and Spy Planes

THE 1970s TURBULENCE AND TERROR

15 Trouble with Henry

16 Disaster at Kizildere

17 Turmoil on Cyprus

18 Unmasking GCHQ: The ABC Trial

THE 1980s INTO THE THATCHER ERA

19 Geoffrey Prime–The GCHQ Mole

20 A Surprise Attack – The Falklands War

21 Thatcher and the GCHQ Trade Union Ban

22 NSA and the Zircon Project

AFTER 1989 GCHQ GOES GLOBAL

23 From Cold War to Hot Peace – The Gulf War and Bosnia

24 The New Age of Ubiquitous Computing

25 The 9/11 Attacks and the Iraq War

26 From Bletchley Park to a Brave New World?

Appendix 1

Appendix 2

Appendix 3 – GCHO Organisation in 1946

Appendix 4 – GCHO Organisation in 1970

Appendix 5 – GCHO Organisation in 1998

Notes

Bibliography

Index

Acknowledgements

By the same author

About the Publisher

Sigint and Comsec Locations in the UK

1 Adastral Park, Martlesham Heath, Suffolk, BT Research Laboratories, 1975–

2 Beaumanor/Garats Hay, Leic., post–war Army sigint base & Special Projects Agency, 1945–94

3 Bletchley Park; this remained a sigint training site after the war until 1985

4 Boddington, Glos, (RAF) military communications unit working with GCHQ

5 Bower, Bowermadden near Wick, listening station, closed 1975

6 Brawdy, Haverfordwest, Wales, 14 Signals Regiment (electronic warfare)

7 Brora, Sutherland, listening station, closed 1984

8 Capenhurst Tower, Cheshire, intercepting telephone traffic to Ireland, 1990–98

9 Cheadle, Staffs, (RAF) listening station, closed 1996

10 Cheltenham (Oakley and Benhall); GCHQ moved to the twin sites between 1952 and 1954

11 Chicksands, Beds, NSA/USAF until 1994, then UK Defence Intelligence & Security Centre

12 Cricklade, Wilts, GCHQ experimental radio station

13 Culmhead, Somerset, GCHQ Central Training School, replacing Bletchley, 1985–94

14 Digby, Lincs, main centre for RAF ground sigint and now UK joint services sigint centre

15 Edzell, Brechin, US Navy/NSA site, 1960–96

16 HMS Flowerdown, near Winchester, listening station, closed 1977

17 Gilnahirk, Belfast, listening station, closed 1978

18 Hanslope Park, near Milton Keynes, Diplomatic Wireless Service and DTMS

19 Hawklaw, (Cupar) Fife, listening station, closed 1988

20 Hereford, 264 Signal Squadron supporting 22 SAS

21 Irton Moor, Scarborough, listening station, now GCHQ Scarborough

22 Island Hill, Comber, Northern Ireland, closed 1977

23 Ivy Farm, Knockholt Pound, Kent, listening station

24 Kirknewton, near Edinburgh, US listening station, closed 1966

25 Menwith Hill, near Harrogate, US Army listening station, taken over by NSA 1963

26 HMS Mercury, near Petersfield, naval signals centre, 1941–93

27 Morwenstow, now GCHQ Bude, focused on satellite communications, 1969–

28 Oakhanger, (RAF) control centre for Skynet since 1967

29 Royal Radar Establishment, Malvern, from 1953, later Defence Research Agency

30 Waddington, Lincs, (RAF) Nimrod R1s of 51 Squadron since 1995

31 Watton, Norfolk, (RAF) Central Signals Establishment, 192 Squadron 1945–63

32 Whaddon Manor, Bucks, outstation of Bletchley Park, closed 1946

33 Wyton, Cambridgeshire, (RAF) Comets and Nimrod R1s of 51 Squadron, 1963–95

34 London

Chester Road, Borehamwood, (GCHQ/SIS) factory making radio microphones in the 1950s

Chesterfield Street W1, London office for GCHQ in the late 1940s

Dollis Hill, North London, Post Office Research Station, 1921–75

Eastcote, Harrow; GCHQ moved here in 1946 and some comsec staff remained after 1952

Empress State Building, Earl’s Court, listening station, 1962–94

London Processing Group, St Dunstan’s Hill, City of London, moved to Cheltenham 1975

Northwood Hills, small post–war GCHQ site; Permanent Joint HQ since 1996

Palmer St W1, LCSA headquarters until 1969; also GCHQ’s London office

Note on Terminology

On 1 November 1919, Britain created the Government Code and Cypher School, or ‘GC&CS’, the nation’s first integrated code-making and code-breaking unit. The term GC&CS remained in widespread use until the end of the Second World War.

By contrast, Government Communications Headquarters, or ‘GCHQ’, is a term of uncertain origin. Originally developed as a cover name for Bletchley Park in late 1939, it competed for usage with several other designations, including ‘BP’, ‘Station X’ and indeed ‘GC&CS’. However, the Government Code and Cypher School remained the formal title of the whole organisation in wartime. During 1946, GC&CS re-designated itself the ‘London Signals Intelligence Centre’ when the staff of Bletchley Park decamped to a new site at Eastcote near Uxbridge, although GCHQ remained in widespread use as a cover name. On 1 November 1948, as Britain’s code-breakers began to investigate a further move away from London to Cheltenham, the term GCHQ was formally adopted and has remained in use ever since.

‘Code-breaker’ is also a troublesome phrase. Codes are usually considered to be words substituted for others, often chosen somewhat at random. Typically, the military operations that constituted D-Day in 1944 were code-named ‘Overlord’. By contrast, systems of communication where letters and numbers are substituted in an organised pattern, either by machine or by hand, are referred to as cyphers. Yet the term code-breaker is so frequently applied to the people who worked at Bletchley Park and at GCHQ that this book follows common usage.

The constantly changing names of the Soviet intelligence and security services are especially vexing and so, despite the inescapable anachronisms, the Soviet civilian intelligence service is referred to as ‘KGB’ until 1989, while the military intelligence service is denoted as ‘GRU’. In Britain, the Security Service is denoted here by the commonly known term ‘MI5’ and its sister organisation, the Secret Intelligence Service or MI6, is referred to as ‘SIS’. Ships’ and submarines’ names are italicised, e.g. HMS Turpin. Onshore naval bases and training establishments, e.g. HMS Anderson, are not italicised.

Abbreviations

A-2—US Air Force Intelligence

ASA—Army Security Agency [American]

ASIO—Australian Security Intelligence Organisation

BDS—British Defence Staff, Washington

BfV—West German security service

BJ—‘Blue jacket’ file for signals intelligence or an individual intercept

Blue Book—Weekly digest of comint material for the PM

BND—Bundesnachrichtendienst – foreign intelligence service of West Germany

Brixmis—British Military Mission to the HQ Soviet Army in East Germany

BRUSA—Anglo–American signals intelligence agreement, 1943

‘C’—Chief of the British Secret Intelligence Service (SIS)

CESD—Communications-Electronics Security Department, succeeded by CESG

CESG—Communications-Electronics Security Group

CIA—Central Intelligence Agency [American]

comint—Communications intelligence

comsec—Communications security

CSE—Communications Security Establishment [Canadian]

CSU—Civil Service Union

CX—Prefix for a report originating with SIS

DIS—Defence Intelligence Staff

DMSI—Director of Management and Support for Intelligence in DIS

DSD—Defence Signals Department [Australian], formerly DSB

DWS—Diplomatic Wireless Service

elint—Electronic intelligence

FBI—Federal Bureau of Investigation [American]

GC&CS—Government Code and Cypher School

GCHQ—Government Communications Headquarters

GRU—Soviet Military Intelligence

GTAC—Government Technical Assistance Centre, established in 2000 – later NTAC

IRSIG—Instructions and Regulations concerning the Security of Signals Intelligence [Allied]

JIC—Joint Intelligence Committee

JSRU—Joint Speech Research Unit

JSSU—Joint Services Signals Unit, combined sigint collection units

KGB—Russian secret service

LCSA—London Communications Security Agency, until 1963

LCSA—London Communications-Electronics Security Agency, until 1965

LPG—London Processing Group

MI5—Security Service

MI6—Secret Intelligence Service (also SIS)

MiG—Mikoyan – Soviet fighter aircraft

MoD—Ministry of Defence

MTI—Methods to Improve, sequential five-year sigint programmes at GCHQ

NATO—North Atlantic Treaty Organisation

NSA—National Security Agency [American]

NTAC—National Technical Assistance Centre, previously GTAC

PHP—Post-Hostilities Planning Committee

PSIS—Permanent Secretaries’ Committee on the Intelligence Services

SAS—Special Air Service

SBS—Special Boat Service

SDECE—French intelligence service

Sigdasys—An allied operational sigint distribution system in Germany in the 1980s

sigint—Signals intelligence

SIS—Secret Intelligence Service (also MI6)

SOE—Special Operations Executive

SUSLO—Special United States Liaison Officer based in Britain

TICOM—Target Intelligence Committee dealing with signals intelligence

UKUSA—UK–USA signals intelligence agreements

VHF—Very High Frequency

Y—Wireless interception, usually low-level

Y Section—SIS unit undertaking interception activities

Y Service—Signals interception arms of the three services

Introduction
GCHQ – The Last Secret?

GCHQ has been by far the most valuable source of intelligence for the British Government ever since it began operating at Bletchley during the last war. British skills in interception and code-breaking are unique and highly valued by our allies. GCHQ has been a key element in our relationship with the United States for more than forty years.

Denis Healey, House of Commons, 27 February 19841

‘GCHQ’ is the last great British secret. For more than half a century, Government Communications Headquarters – the successor to the famous wartime code-breaking organisation at Bletchley Park – has been the nation’s largest and yet most elusive intelligence service. During all of this period it has commanded more staff than the Security Service (MI5) and the Secret Intelligence Service (SIS) combined, and has enjoyed the lion’s share of Britain’s secret service budget. GCHQ’s product, known as signals intelligence or ‘sigint’, constituted the majority of the secret information available to political decision-makers during the Cold War. Since then, it has become yet more significant in an increasingly ‘wired’ world. GCHQ now plays a leading role in shaping Britain’s secret state, and in the summer of 2003 it relocated to a spectacular new headquarters that constituted the single largest construction project in Europe. Today, it is more important than ever – yet we know almost nothing about it.2

By contrast, the wartime work of Bletchley Park is widely celebrated. The importance of decrypted German communications – known as ‘the Ultra secret’ – to Britain’s victory over the Axis is universally recognised. Winston Churchill’s wartime addiction to his daily supply of ‘Ultra’ intelligence, derived from supposedly impenetrable German cypher machines such as ‘Enigma’, is legendary. The mathematical triumphs of brilliant figures such as Alan Turing are a central part of the story of Allied success in the Second World War. The astonishing achievement of signals intelligence allowed Allied prime ministers and presidents to see into the minds of their Axis enemies. Thanks to ‘sigint’ we too can now read about the futile attempts of Japanese leaders to seek a favourable armistice in August 1945, even as the last screws were being tightened on the atomic bombs destined for Hiroshima and Nagasaki.3

However, shortly after VJ-Day, something rather odd happens. In the words of Christopher Andrew, the world’s leading intelligence historian, we are confronted with the sudden disappearance of signals intelligence from the historical landscape. This is an extraordinary omission which, according to Andrew, has ‘seriously distorted the study of the Cold War’.4 Intelligence services were at the forefront of the Cold War, yet most accounts of international relations after 1945 stubbornly refuse to recognise even the existence of the code-breakers who actually constituted the largest part of this apparatus.5 Nor did this amazing cloak of historical invisibility stop with the end of the Cold War. In 2004, following the furore over the role of intelligence in justifying the invasion of Iraq, Lord Butler, a former Cabinet Secretary, was appointed to undertake an inquiry into ‘British Intelligence and Weapons of Mass Destruction’. Butler’s report into the workings of the secret agencies was unprecedented in its depth and detail. However, GCHQ is mentioned only once, in the list of abbreviations, where we are told that the acronym stands for ‘Government Communications Headquarters’.6 This is all we learn, for in the subsequent 260 pages the term GCHQ is in fact never used, and the organisation is never discussed. The subject is simply too secret.

Sigint was not simply a Second World War phenomenon. Throughout the twentieth century, Britain’s code-breakers continually supplied Downing Street with the most precious jewels of British intelligence, discreetly delivered in what became known as the ‘Blue Book’. Nicholas Henderson, formerly Britain’s Ambassador to Washington, explains: ‘All Prime Ministers love intelligence, because it’s a sort of weapon…The intelligence reports used to arrive in special little boxes, and it gave them a belief that they had a direct line to something that no other ordinary departments have.’ It was partly for this reason that British Prime Ministers ‘never minded spending money on intelligence’. Signals intelligence also matters to political leaders because it allows them to hear the authentic voices of their enemies. Although Winston Churchill was the most famous recipient of such material, his predecessor, Neville Chamberlain, was also offered some remarkable insights into the mind of Adolf Hitler. In 1939, shortly after the Munich appeasement, Chamberlain was given an intelligence report which showed that Hitler habitually referred to him in private as ‘der alter Arschloch’, or ‘the old arsehole’. Understandably, this revelation ‘had a profound effect on Chamberlain’.7

However, constant exposure to secrets derived from the world of code-breaking, bugging and other kinds of secret listening has the capacity to induce paranoia. Harold Wilson regularly dragged his Private Secretary, Bernard Donoughue, into the bathrooms and toilets of Downing Street. Only there, with the taps turned on full and water sloshing noisily in the basins, did he feel immune to the threat of bugs.8 A top priority for Britain’s technical security specialists during the Wilson years was the installation of the latest scrambler phones at the Prime Minister’s holiday home in the Scilly Isles, so he could speak to Whitehall without fear of interception. Doubtless, Wilson would have been delighted to learn that some of his opponents felt equally oppressed by electronic surveillance. When Ian Smith, the Rhodesian leader, visited London in late 1965 he insisted on having some of the more sensitive conversations with his delegation in the ladies’ lavatory, convinced that this was the one location where British intelligence would not have dared to plant microphones.9

Secret listening terrified friend and foe alike. Harold Macmillan recalled the almost unbearable sense of oppression he felt on his visit to Moscow to see the Soviet leader, Nikita Khrushchev, in 1959. His delegation feared that British codes were compromised, and they were unable to talk freely, even outside in the open air, because of constant technical surveillance. He would have been fascinated to learn that, at the very same moment, Khrushchev and his immediate circle also felt increasingly anxious about KGB microphones, to the extent that they dared not speak freely, even amongst themselves in their own capital.10 In June 1966, to his immense fury, President Tito of Yugoslavia discovered that he was being bugged by his own security chief. ‘Concealed microphones have been installed everywhere,’ he exclaimed angrily to a friend: ‘Even my bedroom!’11

The supreme example of the way in which eavesdropping could have political consequences was the Watergate scandal, which gradually brought about the downfall of President Richard Nixon between April 1973 and July 1974. Nixon had used a team of former CIA operatives known as ‘The Plumbers’ to burgle and bug premises used by the Democratic Party. Not everyone was shocked. In 1973, Britain’s Prime Minister, Edward Heath, made a visit to China. Mao Tse-tung asked him, ‘What is all this Nixon nonsense about?’ Heath asked what he meant by ‘nonsense’. Mao replied: ‘Well, they say he bugged his opponents, don’t they? But we all bug our opponents, don’t we, and everybody knows it? So what is all this fuss about?’12 Others took bugging in their stride. When Tony Blair visited India in October 2001, his security team found two bugs in his bedroom, and reported that ‘they wouldn’t be able to remove them without drilling the wall’. Blair ‘decided against making a fuss’, and quietly moved to another room.13

Eavesdropping and code-breaking are certainly nothing new. Even in medieval times the crowned heads of Europe had recourse to secretive ‘black chambers’ where encyphered letters from diplomats were intercepted, opened and decoded in order to produce intelligence. However, the modern-day GCHQ owes its origins to the arrival of the radio and the enormous impact of science upon methods of fighting during the Second World War. It was the struggle against Hitler that revolutionised the importance of intelligence from encyphered radio messages. Blitzkrieg and surprise attack were the hallmarks of a new style of warfare that arrived in the late 1930s. The sheer speed of war now meant that secrets smuggled under the coat collar of a traditional human spy were no longer of much use to commanders. The code-breakers of Bletchley Park were the perfect answer, offering intelligence in ‘real time’ from intercepted enemy signals. In some cases, messages sent from Hitler to Rommel in the Western Desert were decoded and arrived on Churchill’s desk before they were read by their intended recipient. Soon, Bletchley Park presided over machine-based espionage on an industrial scale.

With the onset of the Cold War, ‘sigint’, as it had become known, seemed equally important for a dangerous new era of nuclear confrontation. Atomic weapons and equivalent breakthroughs in biological and chemical warfare, together with ballistic rockets such as the V2, against which there was no defence, were the new currency of conflict. World leaders were required to comprehend strange new threats and the accompanying possibility of devastating surprise attack – which Lord Tedder, the British Chief of the Air Staff, called a potential ‘nuclear Pearl Harbor’. The precarious world of early warning, deterrence and ‘targeting’ had arrived. Military chiefs demanded better intelligence, and concluded that global sigint coverage was indispensable to the Western allies. By the mid-1950s, Britain’s code-breakers had abandoned their nissen huts at Bletchley Park for new accommodation in Cheltenham, the distinctive radomes and satellite dishes of which became an integral part of the Cold War landscape.14

Ironically, the story of GCHQ after it entered ‘peacetime’ in 1945 is very much about military operations, and even war. Britain’s vast sigint programme was managed by GCHQ, but run in cooperation with the armed services, which used their bases, ships and aircraft to collect the raw enemy signals. As this book reveals, GCHQ sat at the centre of a spider’s web that consisted of many other hidden organisations, both civil and military, which helped it collect signals intelligence. Many of its stories intertwine closely with Britain’s long legacy of small wars and guerrilla conflicts in locations such as Korea, Malaya, Borneo, Aden and the Falklands. GCHQ’s operations also involved hair-raising confrontations with the Russians. Britain ran secret submarine spy missions designed to gather signals intelligence from the Russian fleet. Specially converted submarines entered the protected harbours of the Russian Navy and rose precariously beneath cruisers to within six feet of their electronic quarry. Submarines that were sent on sigint missions – known to their anxious crews as ‘Dodgies’ or ‘Mystery Trips’ – were detected off Murmansk and pursued by Russian destroyers with depth charges. GCHQ’s ocean-going activities have been a well-kept secret, but some British submariners still bear the scars of this secret signals war in the far north.

Code-breaking is sometimes depicted as highly technical – more ‘Billion Dollar Brain’ than James Bond – and therefore perhaps a little dull. But much of the GCHQ story involves dramatic incidents experienced by individual sigint operators in forward locations, including in submarines and aircraft. However it was done, gathering sigint almost always involved a three-stage process. First, someone had to listen in to and record the intercepted message. Throughout the Cold War this person was often the Godforsaken GCHQ ‘operator’ who sat for eight hours at a time in front of a rack radio made by Racal. With headphones on and the volume turned up to ‘max’ he or she endured the freezing cold of the German winter and the unbearable heat of the Iraqi summer. Once the message was captured it was passed back to Cheltenham for processing. If it was in code, it might be given to X Division, a section staffed by ‘boffins’ with vast computers whose power far outstripped that available to ordinary scientists. Finally, intelligence analysts would try to compose the resulting material into useful summaries. Stamped with an excruciatingly high security classification, it was then circulated to Cabinet Ministers, defence chiefs and senior policy-makers. Often, only a few hours after they had been read by the ‘high-ups’, the summaries were whisked away in ‘burn-bags’ and consigned to vast incinerators to protect their secrecy.

GCHQ is also synonymous with the mysterious international network known as ‘Echelon’, run by British and American intelligence. Echelon is the world’s largest information ‘vacuum cleaner’, drawing in huge amounts of communications – an estimated five billion intercepts every day. Yet much of what we have come to believe about this network is wrong.15 The Anglo–American sigint relationship is often portrayed as a cosy affair of affable, pipe-smoking professor types. In fact, the politics of intelligence was often opportunistic and harsh. Secretly, the British and Americans worked together to read the traffic of their own minor allies, including France and West Germany. Even at the top, relations between the two main partners, Britain and the United States, could turn nasty and involved sharp disagreements.

What bound Britain and America together in the world of signals intelligence was realism, not romanticism. Anglo–American intelligence cooperation was about trading ‘terrain for technology’. America had its own vast code-breaking organisation, the National Security Agency (NSA), with infinitely more resources than the British. However, the American code-breakers needed remote outposts in Britain’s ‘residual empire’ at which to base their listening stations, and they rewarded GCHQ handsomely with access to remarkable technology. Some locations, such as Cyprus, were so important to the collection of sigint that UKUSA actually helped to shape the international politics of the region. In 1974, faced with a financial crisis, the British government formally decided to withdraw from its bases in Cyprus in order to save money. Within days, Washington told London that this decision was not acceptable and they must stay. The reason was simple. The sigint bases that allowed America to listen in to the Middle East were quite indispensable. In 2009, more than thirty years after the British government’s decision to withdraw from Cyprus, the sigint bases are still there, and have grown considerably in size.

Cold War espionage activity enjoyed a high profile. British defectors such as Guy Burgess and Donald Maclean hit the headlines in the 1950s. The 1960s opened with the shooting down of the American U-2 spy plane piloted by Gary Powers, the CIA’s fiasco at the Bay of Pigs and the Profumo affair. Yet GCHQ managed to avoid the glare of unwelcome publicity until the last decade of the Cold War. Its journey from the shadows into the spotlight only began in 1976, when the radical journalist Duncan Campbell revealed its intelligence operations on Cyprus in an article in Time Out magazine. This led to the infamous ‘ABC trial’, at which Campbell and his associates were prosecuted under the Official Secrets Act. Thereafter, GCHQ’s hopes to return to obscurity were dashed by the Geoffrey Prime affair in 1982. Prime, who revealed the innermost working of America’s latest multi-billion-dollar sigint satellite programme to the Soviets, was one of the most damaging moles ever recruited from inside British intelligence. Just as the Prime case subsided, any hopes of a return to anonymity were obliterated by Margaret Thatcher’s controversial decision to ban trade unions at GCHQ.

Expensive technical agencies such as GCHQ and America’s NSA were obvious targets for cuts at the end of the Cold War. At the same time, both agencies were struggling to cope with the pace of the global information-technology revolution, that had made access to high-grade encryption easy for the private individual. All this, together with the exponential growth in internet traffic, threatened to make the work of GCHQ and NSA impossibly difficult. Soon the world was sending several million emails a second, and not even the great sigint leviathans could read them all. The days of the super-secret sigint agencies seemed numbered. However, in the 1990s Britain’s prominent role in the wars in Bosnia and then Kosovo reminded government that the need for sigint is perennial. In these Byzantine conflicts, the radio experts at Cheltenham were never quite sure which of the many different former Yugoslavian factions their various friends and allies were supporting.

Bitter conflicts such as Bosnia helped to convince Whitehall and Westminster that GCHQ was worth new investment. In 1996, under the direction of Sir David Omand, GCHQ began to develop plans for a remarkable new intelligence headquarters that quickly became known as ‘the Doughnut’ owing to its circular design. The intention was to bring all the staff together under one roof for the first time. Absorbing no less than fifteen miles of carpet and several hundred miles of fibre-optic cabling, ‘the Doughnut’ constituted the largest secret intelligence headquarters outside the United States. However, by the time it was completed in 2003, it was already too small. GCHQ had by then undergone a crash expansion following the 9/11 terrorist attacks. Its employees, now numbering more than 5,200, were soon ‘hot-desking’. A shanty town of subsidiary buildings is already springing up around the new headquarters.

Today, in somewhat cramped circumstances, GCHQ struggles with some of the most difficult issues of the twenty-first century. Not only is it the leading edge of Britain’s struggle against al Qaeda, it is also involved in fundamental issues of freedom and privacy that will shape the future of our society. Over the last decade, Britain has engaged with global e-commerce and finance more enthusiastically than perhaps any other country in the world. Our porous electronic borders present their own enormous problems. Globalisation, and in particular the global communications revolution, has brought many benefits, but it has also allowed miscreants to communicate and organise anonymously. The need for GCHQ to monitor both terrorists and organised crime means that the distinction between domestic and foreign communications has less meaning than it once had. GCHQ used to be a wholly outward-looking foreign intelligence service, but this is no longer the case.

Who will rule the internet? Will ordinary citizens be allowed genuinely confidential communication? Would ID cards erode our privacy or extend our security? These are some of the questions that GCHQ ponders daily at the beginning of the second decade of the twenty-first century. Britain is already one of the most watched societies in the world, and some would argue that it is now addicted to surveillance. In 2008, Britain announced a £12 billion project to modernise the interception of telephone calls and email. The following year GCHQ announced a remarkable project entitled ‘Mastering the Internet’ that collects the details of Britain’s communications and internet traffic for security purposes. Even Britain’s Director of Public Prosecutions thought things had gone too far. Tasked with taking the lead on technological aspects of intelligence, GCHQ now finds itself at the centre of controversies that are of immense public importance. Accordingly, the time is ripe to trace GCHQ’s long and secretive journey from the nissen huts of Bletchley Park – via the Cold War – towards what now looks increasingly like a Brave New World.

THE 1940s BLETCHLEY PARK AND BEYOND

1 Schooldays

‘How wonderful!’ I said. ‘Do you mean we’re overhearing Portsmouth ships trying to talk to each other – that we’re eavesdropping across half South England?’

‘‘Just that.’

Rudyard Kipling, ‘Wireless’, 19041

In December 1902, Guglielmo Marconi made history by sending the first wireless radio message across the Atlantic. Remarkably, only two years later, Rudyard Kipling foretold the possibility of exploiting such radio messages to gather intelligence. In 1904 he published a short story entitled ‘Wireless’ that focused on intercepting communications sent from Morse equipment on board Royal Navy ships off the Isle of Wight. Kipling is thought of as a quintessentially late-Victorian author, but here he looks to the future, more in the manner of H.G. Wells, as his characters fret over technical matters such as induction and radio frequencies. To the readers of this fictional first instance of radio interception, the process seemed utterly magical. The Morse instrument ‘ticked furiously’, and one of the listening party observes that it reminds him of a séance, with ‘odds and ends of messages coming out of nowhere’. His companion retorts that spiritualists and mediums ‘are all impostors’, whereas these naval messages that they are eavesdropping on are the real thing.2

Kipling’s ‘Wireless’ is the first public discussion of the secret business of signals intelligence, or ‘sigint’. The magical process of extracting information from the ether would be one of the twentieth century’s most closely guarded secrets. Initially, producing ‘sigint’ only required equipment that would allow a third party to eavesdrop on a conversation broadcast by a radio transmitter using ‘wireless telegraphy’, but as this possibility became more widely known, communicators often resorted to using cyphers to keep their messages private. Thereafter, producing sigint usually required skilled listeners to capture the message and then a team of code-breakers to unscramble it. If the message was sent by cable rather than wireless, the listening-in process could be no less difficult than the code-breaking, or ‘decyphering’.

What did Britain’s code-breakers make of Kipling’s public airing of their black arts? The simple answer is that there were none to ask. Indeed, there had been no British code-breakers for more than fifty years. In the distant past, Britain had possessed a ‘black chamber’ in which skilled ‘cryptanalysts’ had broken the codes contained in diplomatic correspondence and private letters. These arcane skills resided in the ‘Secret Department’ of the Post Office. However, in 1847 this was exposed in a scandalous episode when the House of Commons heard that the Home Secretary had ordered the interception of the private correspondence of the heroic Italian nationalist in exile, Giuseppe Mazzini. Shocked Members of Parliament ordered an inquiry, leading to the closure of the ‘Secret Department’, just as the telegraph initiated what we now understand as a Victorian communications revolution. By 1904, Britain had been without a code-breaking centre for more than half a century3

The immediate origins of MI5 and its sister service SIS (often known as MI6) can be traced to scares about German espionage in 1909. But British code-breaking was not revived until the very eve of the First World War. On 2 August 1914 the British Army set up a secret code-breaking section called MIlb. Soon, specialist Army units at various locations in Europe and the Middle East were busy intercepting German radio communications. One of the largest sites was the intercept station in Mesopotamia. In December 1916 the military code-breakers of MIlb were given a fabulous Christmas present when the drunken chief of the German signals organisation in the Middle East sent all his Radio Operators a seasonal greeting using the same obvious formula in no fewer than six different codes. Up until that point the British had only been able to read one of these codes, but with these clues they could read all six. In the First World War, the Second World War and again in the Cold War, poor discipline by the human operators often proved to be the great weakness in otherwise impregnable cypher systems.4

The Royal Navy code-breakers, who had established themselves in the Admiralty’s ‘Room 40’, achieved even greater success. Famously, they broke the ‘Zimmermann Telegram’, a message sent from the German Foreign Minister, Arthur Zimmermann, suggesting an alliance between Germany and Mexico against the United States. As an inducement, Mexico was to be offered the return of her lost territories in Texas, New Mexico and Arizona. These revelations, made public in March 1917, were central in bringing the United States into the First World War on the side of Britain and France. The American entry into the war, together with a tightening blockade, persuaded Germany to seek an armistice the following year. The code-breakers of Room 40 celebrated with champagne. There are few more significant examples of the direct impact of code-breaking upon international relations.5

In 1919 the British government’s Secret Service Committee, chaired by Lord Curzon, the rather formidable Foreign Secretary, recommended that a unified peacetime code-breaking agency should be created. This involved the difficult merger of two quite separate organisations. The head of the Army code-breakers, Major Malcolm Hay, was awkward and argumentative, while his naval equivalent, Commander Alastair Denniston, proved to be suave and diplomatic. Denniston secured the job as chief of a new combined code-breaking organisation, which initially consisted of around two dozen intelligence officers and a similar number of clerical staff, and found himself installed in splendid accommodation at Watergate House in The Strand, next to the Savoy Hotel. Formed on 1 November 1919, the new organisation was given the name ‘Government Code and Cypher School’, or GC&CS, which was not inappropriate, since the leading code-breakers devoted a great deal of time to the patient training of new initiates.6 Both during the First World War and in the interwar period about half the staff of GC&CS and its predecessors were women, mostly in the clerical grades.

Almost immediately, GC&CS adopted a disingenuous description of its duties that would remain in place until the 1980s. Publicly, its functions were described as merely defensive; in other words, it was to assist in the provision and protection of codes and cyphers used by government departments. However, its more secret duty was to give priority to offensive activity, namely attacking the cypher communications used by foreign powers. GC&CS gradually shifted its focus to diplomatic traffic, and at the suggestion of Lord Curzon it was transferred to the control of the Foreign Office. It seemed natural that within the Foreign Office structure it should be placed under the supervision of Britain’s traditional overseas intelligence service, SIS, which recruited human spies. But a subliminal naval influence remained. The talented Chief of SIS, Mansfield Cumming (known within the organisation as ‘C’, the name by which the head of SIS would continue to be called), was a former naval officer. Cumming died in harness in 1923 and was succeeded by another sailor, the former head of Naval Intelligence, Hugh ‘Quex’ Sinclair. Naval intelligence and naval signals officers continued to exercise a profound influence on GC&CS and its successors as late as the 1970s.

The means by which Britain collected its intelligence was changing. During the First World War, much of its intelligence work had involved overhearing military wireless messages by means of receiving stations scattered around Europe, the Mediterranean and the Middle East. The shift to diplomatic traffic meant undertaking more work on encyphered diplomatic telegrams sent by cable. Each country had teams of code clerks who carefully encyphered diplomatic messages before they were sent by telegram using a worldwide network of cables. Although government cable censorship had officially ended in 1918, a private arrangement meant that all the commercial cable companies secretly handed over their traffic to GC&CS for copying. Most of the foreign embassies in London used cable companies to send their encyphered messages, and British dominance of international telecommunications networks meant that many of the world’s messages travelled over British cables at some point. Private companies such as Standard Cable & Wireless Ltd were almost an integral part of the worldwide British sigint system. This secret state-private network remained hidden until it was exposed by the journalist Chapman Pincher in February 1967 in the Daily Mail under the headline ‘Cable Vetting Sensation’.7

In 1925 both SIS and GC&CS were moved into Sinclair’s new secret service headquarters at Broadway Buildings, opposite St James’s Park tube station, which its occupants thought ‘more dingy than sinister’. The walls of the corridors were painted dark brown to a height of about four feet from the floor, and the ancient lifts moved between the many storeys with a slow clatter. The code-breakers were given the third floor. From here, the sigint product, which consisted of the verbatim text (or sometimes summaries) of the messages of foreign governments was distributed around Whitehall in files with special blue jackets that became known as ‘BJs’. GC&CS worked on the cyphers of many countries in the interwar period, including those of France, the United States and Japan, since they all shed light on international affairs; but the most important were those of Russia.8

Both MI5 and SIS, together with intelligence officers from the three armed services, were obsessed with the threat from Bolshevik Russia in the interwar period. GC&CS followed suit. There were good reasons for making Moscow the pre-eminent target. Bolshevik agents were actively seeking to subvert the British Empire, and sigint produced operational intelligence that could be used to thwart these plots. Alastair Denniston enjoyed a major advantage, having recruited Ernst Fetterlein, the Tsar’s leading code-breaker, when he fled Russia after the Revolution of 1917, and in the 1920s GC&CS was successfully reading Soviet diplomatic cyphers. Several times during that decade the British government directly accused the Soviets of underhand activities in London, making use of these intercepts and referring to them openly. In 1923, for example, Lord Curzon publicly quoted Soviet messages intercepted by GC&CS stations in India. The Soviets responded by changing their cyphers, but Fetterlein simply broke them again.9

However, in May 1927, a year after the General Strike, a disastrous row erupted over secret support from Moscow for the strikers and the distribution of subversive propaganda in Britain. A veritable centre for Soviet subversion was being run under the cover of its Trade Mission, located in the Arcos building in Moorgate. The building was raided on 12 May, but advance warning allowed the Soviets to destroy most of the incriminating material. The Prime Minister, Stanley Baldwin, was embarrassed. He ardently desired to break off relations with Moscow, but having failed to garner any incriminating evidence from the Arcos raid, he turned to the priceless intercepts provided by GC&CS. To the dismay of the code-breakers, Baldwin and his Foreign Secretary, Neville Chamberlain, read out four decyphered Soviet telegrams in Parliament in order to make their case. Alastair Denniston was especially bitter about this flagrant compromise of GC&CS secrets.10

Henceforth, the Soviets changed their cyphers and deployed more secure systems for communications with diplomatic and commercial missions overseas, including their intelligence stations. They now used the ‘one-time pad’ for their more important communications. The one-time pad was a breakthrough system created by an American army officer, Major Joseph Mauborgne, during the First World War and widely adopted by other powers. It involved using a sheet of random numbers to encypher a message. Each letter in the message was given a number. Each number was then added to another from a stream of random numbers taken from a sheet on the one-time pad. The result was a sheet of text that consisted simply of groups of five numbers, one after another. Recipients could decode the message if they possessed the same sheet from the same one-time pad. If that sheet was used only once – hence the name – and for a single message, the lack of repetition prevented decryption. In short, the code was unbreakable. The disadvantage was that it was slow and cumbersome, and therefore it was reserved for high-grade secrets. Moreover, vast numbers of pads with lists of random numbers were required. No country, not even the security-obsessed Soviet Union, could send all its communications by this means.11

Nevertheless, after 1927, few Soviet diplomatic messages were being read by GC&CS. The only high-grade Soviet traffic that was decyphered were the messages of the Comintern, the part of the Soviet Communist Party that dealt with relations with Communist parties overseas. This effort was led by John Tiltman, a brilliant major from the Indian Army who had been running a small but successful interception effort in north India during the 1920s. In 1929 he was brought back to London to lead an expanded operation against Comintern communications (which were code-named ‘Mask’). This allowed the British government to learn of the secret subsidies paid by Moscow to the Communist Party of Great Britain and its newspaper, the Daily Worker. It also contributed to important successes against major Comintern agents in imperial outposts and international centres such as Singapore and Shanghai.12

Faced with the real threat of active subversion throughout the British Empire by the Comintern, GC&CS paid limited attention to military matters or the rise of the Axis until the mid-1930s. Germany, Italy and Japan were a remarkably low priority. Admittedly, a small naval section of GC&CS had been set up in 1925, and its most important work was done overseas by naval officers like Eric Nave, based in Hong Kong. From here they had ample practice at following military operations, because of the extensive fighting in Manchuria during the 1930s. Italy’s attack on Abyssinia in 1936 provided a new target for British code-breakers in the Middle East, located at sites such as Habbaniya in Iraq and Sarafand in Palestine. Remarkably, and despite the growing importance of air power, GC&CS only developed an RAF section in 1936, under Josh Cooper, a young and talented code-breaker who had joined the organisation a decade earlier with a First in Russian from King’s College London.13

Cyphers were important to the Axis military powers. One-time pads were slow and cumbersome. Moreover, they were out of step with the emerging new methods of warfare. Blitzkrieg, for example, required armoured forces to move forward at lightning speed, coordinating their activities with artillery and air support. So the pressure was on to find a way of making the growing volume of military radio traffic unintelligible to the enemy. Most developed countries turned to cypher machines to make their immense volumes of traffic secure.14 Complex cypher machines had been pioneered by banks and businesses – banks had long used fairly simple cyphers to keep commercial matters secret. In the 1920s, the German military adapted a Dutch invention to produce the Enigma cypher machine as an alternative to laborious hand cyphers. In fact, the first Enigma machines were sold commercially, and were widely used by banks and businesses. Enigma was what we now recognise as a ‘commercial off-the-shelf solution’ to a difficult military problem.15

The Enigma machine itself looked like an early typewriter in a square wooden box, but with a keyboard set out in alphabetical order rather than the traditional ‘QWERTY’ arrangement. As each letter key was depressed a set of lights that corresponded to the alphabet lit up, seemingly at random. The innovation was the rotors, which looked like fat metal wheels, embedded in the top of the machine. These rotated and scrambled the message in a highly unpredictable way. There were initially three – later four – rotors, with twenty-six positions relating to the letters of the alphabet. These moved round in a stepping motion that generated a cypher with an enormous number of possibilities. Moreover the complex nature of the rotation caused subtle changes in the stream of material, creating substantial headaches for any would-be code-breaker. The Germans were not alone in developing cypher machines. The British and Americans developed similar devices, respectively called the Typex and Sigaba.16

Critical to the breaking of Enigma was assistance from the secret services of France and Poland. French intelligence employed a lugubrious German agent called Hans Schmidt, who worked in the German military cypher department. Fond of the finer things in life, which the French secret service supplied to him in abundance, Schmidt divulged many technical documents about Enigma, including messages in both clear and encyphered text. He was later betrayed, and would commit suicide using cyanide procured for him by his daughter. By 1938 these secrets were being shared with the British through ‘Biffy’ Dunderdale, the SIS station chief in Paris. However, when the French gave the British material on German Air Force communications a further secret was accidentally revealed, namely that the French were also working on Enigma in collaboration with the Poles. In January 1939 Alastair Denniston took two of his top code-breakers, Hugh Foss and Dilly Knox, to Paris to meet their French and Polish opposite numbers. Eventually they discovered that the Poles had completely reconstructed the German version of the Enigma machine.17

Remarkably, by 1938 the Polish code-breakers were able to read the majority of German Army Enigma messages. The Polish breakthrough had been to train professional mathematicians to help them, together with the use of a primitive processor called the ‘bomba’ or ‘bombe’ – so named because of the alarming ticking noise it made – to find the rotor settings. One of their first ‘bombes’ was a weird contraption that consisted of no fewer than six Enigma-type machines wired together to provide rapid processing of possible solutions. Polish resources were limited, and by late 1938 new advances in the Enigma machine were running ahead of the ability of the Poles to do their calculations. But the precious secrets that the Poles taught the British were enough to continue the unravelling of Enigma. The timing was an extraordinary stroke of luck, since the talented Polish cypher bureau was within two months of being broken up by the coordinated German–Soviet invasion of Poland in the autumn of 1939. Before the Polish secret service was forced to flee Warsaw, its agents had achieved the remarkable feat of stealing several examples of the military Enigma machine from the German factory where they were made.

In the late 1930s, Britain lived in the shadow of the aerial bomber. Following the tragic fate of the Spanish town of Guernica in the spring of 1937, the presumption was that the first few days of the approaching war with Germany would bring untold destruction from the air, levelling the cities of Europe. By the Munich Crisis of 1938, Whitehall had begun to make emergency preparations. Admiral Hugh Sinclair, the Chief of SIS, was busy looking for alternative wartime accommodation away from London for both SIS and GC&CS. He soon settled on a country house, Bletchley Park, near Milton Keynes in Buckinghamshire, as an ideal location for the code-breakers. Much has been made of Bletchley Park’s proximity to Oxford and Cambridge, but in fact the availability of good trunk cable communications was the dominant consideration. Bureaucratic bickering now erupted. Although GC&CS was run by the Foreign Office, its relocation was considered to be war contingency planning, so the diplomats insisted that the military pay the bill. Predictably, the War Office insisted that GC&CS was nothing to do with it, and emergency relocation for Britain’s most valuable wartime asset stalled. In the end, Hugh Sinclair bought Bletchley Park with his own money, paying over £7,500 (more than £330,000 at today’s prices). This remarkable act of generosity allowed the first wave of evacuated staff to arrive at Bletchley on 15 August 1939. Sinclair’s largesse did not stop there. He acquired a top chef from London to provide food to the code-breakers in a restaurant in the main hall, complete with full waitress service.18

The emphasis at Bletchley Park was distinctly military. The main body of GC&CS was initially broken up into Naval, Military and Air Sections and allocated to the ground floor of the main house, while SIS was given the top floor, indicating that it still ruled the roost. On the periphery, an ever-growing collection of numbered wooden huts – including the famous Hut Three and Hut Six – were being constructed. Particular activities were associated with each hut: typically, the core of the Enigma problem was worked on in Hut Six, while its exploitation for intelligence purposes was undertaken in Hut Three. One former code-breaker recalls that the main house was soon ‘too small for more than a handful of top brass and their immediate acolytes’. So Bletchley Park’s considerable garden, with its rosebeds and delightful maze, gradually disappeared beneath the expanding penumbra of temporary structures.19 The shadow of the bomber even reached out to Bletchley Park. The radio transmission infrastructure involved elaborate aerials which had the potential to give away the site’s location from the air. Accordingly, Bletchley Park’s own radio station was moved to nearby Whaddon Hall. As the operation gained momentum, other nearby premises were absorbed. Elmers School, a neighbouring boys’ boarding establishment, was requisitioned for the GC&CS Diplomatic Sections.

Bletchley Park was Admiral Sir Hugh Sinclair’s last bequest to Britain’s sigint community. Through the early autumn of 1939 it was clear that he was terminally ill with cancer. His deputy and heir apparent, Stewart Menzies, was not regarded as a great brain, and indeed despised intellectuals. Sir Alexander Cadogan, the Permanent Under-Secretary at the Foreign Office, vigorously resisted the idea that Menzies might succeed Sinclair, and argued for someone from outside SIS to shake the organisation up. Senior SIS officers, however, did not want ‘a new broom at this critical stage’.20 Cadogan noted in his diary, ‘I am not satisfied that Menzies is the man,’ but Menzies did have a crude talent for furthering his own ambitions, which he soon demonstrated. On Sunday, 5 November he came to see Cadogan bearing the sad news of the death of Sinclair the previous day. Cadogan noted that he ‘gave me a sealed letter from “C” recommending him (M[enzies]) as successor’.21 Lord Halifax, the Foreign Secretary, pressed for Menzies, who was finally accepted as the new Chief on 28 November.22

The arrival of Menzies was a problem for Bletchley Park because the code-breakers were still subordinate to SIS. Under Menzies the administration of SIS was ‘chaotic’, and its headquarters was in ‘a state of upheaval’ throughout 1940.23 Cadogan maintained his view that Menzies was a mentally disorganised intriguer who devoted more time to protecting the interests of SIS than to serious intelligence-collection. Typically, in March 1941, after Cadogan had met Menzies and the Directors of Intelligence of the three armed services, he recorded in his diary: ‘“C” as usual, a bad advocate on his own behalf. He babbles and wanders, and gives the impression he is putting up a smokescreen of words and trying to put his questioners off the track.’24 Cadogan longed to see a thorough overhaul of SIS, which he regarded as an organisational basket-case. However, as the war dragged on, he had less and less time for the politics of intelligence.25 Quite understandably, SIS wanted to keep all code-breaking under its wing, since it was a form of foreign intelligence-gathering. Menzies was also adamant that he should retain personal control over Ultra.26 If possible, he preferred to take this material to Churchill personally, basking in its reflected glory. But he did not know how to manage Bletchley Park, and as a result it was under-resourced.27 In the words of one SIS contemporary, Menzies regarded anything to do with personnel or administration as ‘dirty work’, and would go to considerable lengths to avoid it.28

Bletchley Park may have been chaotic, but it was a creative and innovative chaos that allowed the code-breakers to make a fresh start in the Buckinghamshire countryside.29 The head of GC&CS, Alastair Denniston, spent the autumn of 1939 making detailed war preparations. His task was to find new cryptographers to fill out the ranks of Bletchley Park. His valuable contacts with the Poles and their success with the ‘bombe’ had led him to realise that he not only needed more code-breakers, he also needed mathematically-inclined individuals. Most of the current inhabitants of GC&CS were linguists with a penchant for Latin and Greek. He now needed people who loved maths and machines, and in September 1939 he was actively scouring the high tables of Oxbridge colleges for talent. The brilliant new mathematicians he recruited included Gordon Welchman from Trinity College, Cambridge, who would run the heart of the code-breaking operations in Hut Six. He brought with him Stuart Milner-Barry from the same college, who was the chess correspondent of The Times and who eventually took over as head of Hut Six. In turn, Milner-Barry brought fellow members of the British chess team, Hugh Alexander and Harry Golombek, to Bletchley.30

These arrivals came not a moment too soon. Hitler’s attack on Poland had tipped Europe into all-out conflict, and Bletchley Park was now a fully operational war station. The pressure was on to make progress against Enigma. The most brilliant mind engaged in this task was Alan Mathison Turing, who made an early and important contribution. Despite understanding the abstract problems of Enigma some months into the war, GC&CS was having difficulty in breaking any real Enigma messages, and was not delivering much product. To have examples of the machine was not enough, since the security of the messages it sent depended on the ‘key’, in other words the settings of the machine, which changed each day. Turing was sent to see the remnants of the Polish code-breaking team, now residing near Paris, to try to work out what the British were doing wrong. The Poles explained that the British had failed to think through the way in which the wiring was attached to the rotors of the Enigma machine.

In early 1940, with this further helpful shove from its allies, Bletchley Park began breaking substantial amounts of Enigma traffic. There were many different Enigma cyphers, and to distinguish them, they were colour-coded. In February 1940, Bletchley Park began breaking ‘Red’, which was an invaluable system used for liaison between the German Army and the Luftwaffe. Periodically, a change to a German cypher system would cause the British code-breakers to lose it for a while, and quite often recovering it depended on second-guessing the lazy habits of the operators. German overconfidence in the improved Enigma machine led to basic mistakes that greatly simplified the task of those whose objective was to tease out the rotor setting for each day.31

By early 1941, the flow of material from the breaking of Enigma was impressive. The intelligence from Bletchley Park was circulated on a very select basis, and was marked with the code word ‘Ultra’ to denote the extremely high level of security attached to the material. Menzies showcased his triumph by taking senior figures from Whitehall on day trips to Bletchley Park. On 11 January it was the turn of Alexander Cadogan. He noted in his diary:

Cold but thawing. Had a rush at the FO till 11, when I left with Menzies for Bletchley. Got there about 12.30. Very Interesting – I should like to spend a week there so as to try and understand it. A charming young Cambridge professor of geometry – Welshman [Gordon Welchman] – did his best with me. A good show, I think.32

Others soon made the pilgrimage to the strange mock-Tudor mansion surrounded by temporary huts. On 6 September 1941 Winston Churchill himself, now Prime Minister, stood on a pile of bricks left by some workmen alongside Hut Six and gave an impromptu speech – delivered with deep emotion – about the value of Bletchley Park to the war effort.33

Unbeknown to Churchill, Bletchley Park was in deep crisis. This was partly due to its rapid growth, and partly to the uncertain institutional boundaries that were evolving almost daily. The situation was exacerbated by a complex relationship with the ‘Y services’, the lower-order radio intercept organisations run by the Army, Navy and Air Force that fed Bletchley with captured traffic. Meanwhile the three armed services were themselves vying for increased control over who received the output from GC&CS. This was precisely the kind of complex organisational puzzle that Menzies was ill-equipped to deal with. Matters reached a head in the autumn of 1941, forcing Menzies to appoint a Joint Committee of Control, which included members of both SIS and GC&CS. However, as the historian Philip Davies observes, ‘Like so many of Menzies’ administrative initiatives, the committee proved unequal to the task.’34 There was also a general resources problem. Having made significant inroads into German Enigma traffic, there were simply not enough staff at Bletchley Park to process the vast torrents of accessible German communications. Neither Alastair Denniston nor his deputy, Edward Travis, had the pull in Whitehall to overcome the shortage.35

Churchill was not ignorant of this state of affairs for long. Recalling the Prime Minister’s kind words during his recent visit, the code-breakers resolved to go straight to the top. On 21 October 1941, four of the most brilliant minds at Bletchley Park, Hugh Alexander, Stuart Milner-Barry, Alan Turing and Gordon Welchman, wrote directly to Churchill to beg for more resources, explaining that their work was so secret that it was hard to explain their requirements to those who controlled personnel.36 So secret was their missive that Milner-Barry took the train to London and delivered it personally to 10 Downing Street. Churchill was shocked by these revelations, and demanded ‘Action This Day’. He ordered his military assistant, General Hastings ‘Pug’ Ismay, to ensure that GC&CS had everything it needed, and to report that this had been done.37 As a result, Bletchley Park underwent a further expansion, and more importantly a major reorganisation.38

GC&CS was now divided into two distinct parts, civil and military. The end of the Blitz meant that the civil side, which dealt with economic and diplomatic traffic, could be sent back to London with relative safety. It took up residence in Berkeley Street, partly because the work of attacking diplomatic codes often had to be coordinated with discreet telephone taps on the foreign embassies in London. The military side remained at Bletchley Park. This did not resolve the heated arguments about who controlled the spoils of GC&CS, but it did address the immediate accommodation problems, and created two organisations of a more manageable size. Menzies retained his post as overall Director, but was a notably absentee landlord. Alastair Denniston was sent to London as Deputy Director (Civil), while his talented deputy, Commander Edward Travis, remained at Bletchley as Deputy Director (Services).39 Travis was now the rising star.40

British code-breaking in the early years of the war was not just about the German military secrets revealed through Enigma. Even harder to break than the Enigma machine had been a German teleprinter on-line cypher machine known as ‘Tunny’, used by the German High Command to produce ‘Fish’ messages. On-line cypher machines were especially challenging because they were automatic, and sent a continuous stream of text, much of it dummy material, sometimes offering no obvious start or end points to each message. This went some way to eliminating another weakness of the Enigma machine – its operators, who were prone to human error. To address the problem of ‘Tunny’, the British later built ‘Colossus’, one of the earliest general-purpose electronic machines, and perhaps the first device that might be described as a ‘computer’. Conceived by Professor Max Newman and then developed by Tommy Flowers from the British Post Office research facility at Dollis Hill, this was one of the supreme technical achievements of the war.41

The achievements of the civil side of GC&CS have often been neglected. By 1940 it was analysing not only the diplomatic codes and cyphers of the Axis powers, but also those of more than twenty other countries. These included the Soviet Union, which did not enter the war until it was attacked by Germany on 22 June 1941. The diplomatic communications of quarrelsome allies such as the Free French, or important neutrals such as the Turks and the Spanish, proved as interesting and as useful as those of Germany. Moreover, the traffic of Germany’s allies, such as Japan, could shed a penetrating light on the mindset of Berlin. Throughout 1941 Hitler held regular meetings with Baron Oshima, the Japanese Ambassador in Berlin, often referred to as ‘Hitler’s Japanese confidant’. Japan had its own complex cypher, known as ‘Magic’, produced by a machine called ‘Purple’, and Oshima used it to send detailed accounts of his long conversations with Hitler to Tokyo. ‘Magic’ had been broken by the Americans, and early Anglo–American cooperation on code-breaking ensured that all this was being read in London. Remarkably, Berkeley Street was also working on the cyphers of the United States, which did not join the war until the Japanese attack on Pearl Harbor on 7 December 1941.42

The dramatic events of 1941 transformed the course of the Second World War. Although the Battle of Britain had staved off the possibility of a German invasion, by the summer of 1941 Britain had been fighting for almost two years without a major victory. Therefore, Hitler’s bizarre decision to invade Russia in June 1941, which required the legions of the Wehrmacht to turn east, provided a welcome breathing space. After Japan’s attack on Pearl Harbor, Britain, the United States and Russia found themselves ranged together against the Axis in what was soon called the ‘Grand Alliance’. Welcome as this was, a genuine world war created new dilemmas for the denizens of Bletchley Park, who now confronted the ticklish issue of large-scale Allied cooperation in the business of code-breaking.

2 Friends and Allies

…there is no better analogy than the schoolboy with his stamp collection.

GC&CS, discussing intelligence cooperation with the Russians in 19431

The most secret aspect of Bletchley Park’s wartime work was its dealings with friends and allies. Many have pondered whether the British attacked Soviet codes and cyphers during the Second World War. The official history of British intelligence insists that Churchill ordered this activity to stop in June 1941, following Hitler’s invasion of the Soviet Union, since Moscow had suddenly become an ally.2 However, it is now clear that this is quite untrue. At the end of October 1941, intelligence chiefs were actually discussing the expansion of the sigint organisation in India, which was then dealing with ‘material from Russian, Persian and Afghan sources’. Remarkably, it was not yet working on German traffic.3 Moreover, in January 1942, and again in early 1943, the British and the Americans were discussing the mutual exchange of intercepted material from ‘Slavic nations’.4 Soviet cyphers had been the core business for Britain’s interwar code-breakers, and work on this material never stopped completely during the Second World War.

To understand why, we must cast our minds back to the approach of the war. During the 1930s, GC&CS continued to follow the traffic of the Comintern even after other Soviet systems were lost. This revealed persistent efforts to subvert the British Empire in locations such as India, Malaya and Hong Kong. Indeed, the Soviet Union appeared to be in league with Germany after the Nazi–Soviet Pact of August 1939. It is often forgotten that Poland was invaded by Germany and the Soviet Union together. For a nightmare period between August 1939 and June 1941, many suspected that Germany, Italy, Japan and the Soviet Union would act in uneasy concert, dividing the spoils of the world between them. This was precisely the plan that Germany’s Foreign Minister, Baron Joachim von Ribbentrop, was trying to press upon his irascible master. However, in the end Adolf Hitler’s racist outlook could not tolerate the idea of alliance with the Slavic peoples, and he had always declared his desire for ‘Lebensraum’ in the east.5

Throughout this dangerous period, before Hitler and Stalin turned upon each other, the Soviet Union remained a key intelligence target. SIS even organised a secret squadron to conduct aerial reconnaissance of possible bombing targets deep inside southern Russia, notably the oilfields. GC&CS developed close relations with code-breakers in the Baltic states who were also working on Soviet codes. A month after the outbreak of war with Germany, Clive Loehnis, a naval officer at GC&CS (who would become Director of GCHQ in the 1960s), told Alastair Denniston that additional premises were needed to cope with the increase in the interception of Soviet military traffic, so new buildings were erected at Scarborough.6 With the military chiefs keen to ‘get cracking on Russian traffic’, Denniston began a unique and profitable experiment. In 1939 GC&CS sent a party of British sigint operators to Sweden to work secretly out of the British Embassy in Stockholm, where there was better radio reception from Russia. The creation of this forward listening station was fortuitous, since Stalin embarked on the Winter War against Finland in November 1939, and GC&CS enjoyed a front-seat view of the whole proceedings.7

John Tiltman remained the key figure in the effort against Soviet communications. A colonel in the King’s Own Scottish Borderers, he was noted for his smart uniform, which included tartan trews. However, as the war progressed he came under the influence of the spirit of Bletchley Park, and was often seen in a baggy pullover and green corduroy slacks.8 One of his first duties was to visit Helsinki to conclude a deal with the talented Finnish code-breakers. Britain funded the expansion of the Finnish cryptographic bureau, and supplied it with the latest equipment in return for material on the Soviets. In March 1940, after imposing a series of humiliating defeats on the Soviets, the Finns signed the Moscow Peace Treaty, ceding about a tenth of their territory.9 The sigint deal with the British was unaffected, and indeed in September 1940 its scope was expanded during a visit by Admiral Godfrey, the Director of Naval Intelligence. According to an internal GC&CS history written after the war, ‘The Finns had agreed to supply us with copies of all their intercepts and cryptographic successes, provided that we did the same.’ Preceding the agreement with the Americans by more than a year, this was perhaps Britain’s first comprehensive sigint alliance.10

By March 1940, the interception of Soviet traffic was big business. For the first time, collection began in the Middle East, at Sarafand in Palestine, although it was still sent to India for analysis. Soviet traffic was also being taken at Ismailia in Egypt and Dingli in Malta. Meanwhile, in Hong Kong, other British sigint operators were also listening to the Soviets before they packed up their equipment to move to Singapore in anticipation of a Japanese attack. The surge of Soviet traffic meant changes were required at GC&CS, where an inter-service Soviet section was created to work in close conjunction on naval, military, air, diplomatic and commercial material. After the fall of France in the summer of 1940, evacuated French cryptographers joined the effort on Soviet traffic at GC&CS. A Polish section, based at Stanmore on the northern fringes of London, soon discovered that it was able to listen in to Soviet traffic as far away as Ukraine.11

Ultra had provided Bletchley Park with an intimate picture of the build-up of German forces in the east, prior to their attack on the Soviet Union. As early as January 1941 it was clear that Hitler’s vast armies were being moved eastwards in preparation for some grand project. Yet even with the evidence of many German divisions massing in the east, Whitehall refused to believe that Hitler was mad enough to deliberately opt for war on two fronts. Like Stalin himself, the British Chiefs of Staff believed that this was more likely to be a prelude to a German ultimatum, a bluff in which Hitler would demand the cession of some further territory in Eastern Europe. Throughout early 1941, Stalin believed that all war warnings were self-serving efforts at deception by the West, which sought to provoke a war between Germany and the Soviet Union. Stalin has frequently been ridiculed for ignoring the warning signs of the impending attack, but despite the benefits of Ultra, it was only the month before the fateful date of 22 June 1941 that British intelligence chiefs realised what was about to occur.12

Hitler’s decision to turn east was a fabulous stroke of luck for Britain. At a time when its forces were struggling, this was a most welcome redirection of the main German war effort. Taken together with Pearl Harbor at the end of the year, it is right to regard 1941 as nothing less than the fulcrum of the war. However, Bletchley Park now faced a new problem. Should it pass sensitive intelligence derived from Ultra to the new Soviet ally, which had been a dedicated enemy of Britain since 1917? The idea that two of Britain’s adversaries were about to fight to the death filled most military intelligence officers with ill-disguised glee. Many argued that passing sigint to the Soviets was pointless, since few expected them to hold out later than 1942. Others insisted that not even Ultra could penetrate the fog of self-deception with which Stalin had surrounded himself.13

In the event, Bletchley Park did develop a precarious sigint liaison with the Soviets. When the British Chiefs of Staff despatched a military mission to Moscow, the code-breakers decided to work through it to find out what the Soviets were doing. They began cautiously, asking about ‘low-grade material only’, notably German Air Force three-letter tactical codes. They intended to send an officer from Bletchley, and in the long term even hoped to persuade the Soviets to accept a British Y unit, or forward listening station, that would intercept German tactical messages on their front. In late 1941 the Soviets agreed to a visit from Squadron Leader G.R. Scott-Farnie, who worked on Britain’s Y interception system in the Middle East.14

Scott-Farnie gave the Soviets a good deal of information on low-grade German Air Force systems, but quickly came up against a different culture of intelligence exchange.15 The Soviets adored captured documents, and did not attach much credence to any information that was not supported by such evidence. Once the game of document exchange began, Scott-Farnie discovered that the Soviet approach ‘was precisely that of a horse dealer who enjoys the poste and riposte of a bargain, and they looked at the exchange of documents on an eye for an eye basis’.16

Bletchley now had to decide whether to follow up the Scott-Farnie Mission. Alastair Denniston was ‘full of hesitation because of the continued Soviet retreat before the German onslaught’, but the intelligence directorates of Britain’s three armed services thought it worthwhile. Josh Cooper, who had reviewed the exchanged material, concluded that the Soviets were ‘absolute beginners’ in their work on the German Air Force, but thought they should be shown the RAF Y stations at Kingsdown in Kent and Cheadle in Cheshire to point them in the right direction. If the Soviets were impressed, he added, they might allow a British Y unit to be sent to the Soviet Union.17 In the end, Edward Crankshaw, an Army Y Service officer, was sent out, armed with more barter material in the form of documents. This was to be ‘swapped’ with the Soviet interceptors, since Bletchley Park thought ‘there is no better analogy than the schoolboy with his stamp collection’. By the spring of 1942 Crankshaw was established in the Soviet Union, and was trading his wares.18 However, the greatest success in the Soviet Union was achieved by the Royal Navy. It was running supply convoys to the Russian port of Murmansk, and this justified the setting up of a radio station at the nearby town of Polyarnoe. A small naval Y intercept party was soon attached to it, and began cooperating with the Soviets on low-level German naval communications. This kept going until December 1944, and yielded good material on subjects such as the movements of the German battleship Tirpitz in northern waters.19

The main worry about giving Ultra to the Soviets was the insecurity of their own cyphers – in 1942, Bletchley Park was increasingly aware of the German ability to read a great deal of Soviet operational military traffic in the field. Frederick Winterbotham, who worked on sigint distribution, argued that Moscow simply had to be told about the weak security of its cyphers. However, Winterbotham’s colleagues insisted that it was ‘impossible’ to tell the Soviets, even though he had ‘invented a good cover story’ to explain how they knew.20 The secret truth was that Bletchley Park was collecting second-hand sigint. The Germans were sending their own sigint from the Eastern Front back to Berlin using an Enigma key code-named ‘Mustard’, which in turn was being read by the British. Although much of the sigint obtained from the Soviets was operational, the British also noted that ‘first grade traffic can be read – at least in part’ by the Germans. Some of the German successes had stemmed from a Soviet codebook, ‘OKK–5’, known to have been captured by the Finns and given to the Germans. While the British had struggled to break these codes in the 1930s, the Germans were having more success.21 On 16 June 1942, Nigel de Grey, the Deputy Director at Bletchley Park, stepped in and settled the argument. He noted that Edward Crankshaw, the GC&CS liaison with the Soviets, would soon be returning from Moscow for another visit. He would be ordered to give the Soviets the details of their compromised cyphers and ‘the methods of reading’. This decision probably reflected the fact that, against all predictions, the Soviet forces were hanging on impressively and looked as if they were going to be in the war for some time to come.22

In August 1942, Crankshaw briefed the Soviets on their appalling lack of security, typified by their alarming tendency to use low-grade cyphers for high-grade secrets.23 There was abundant evidence of this in German Air Force Enigma, but Crankshaw only hinted at it by ‘somewhat tenuous means’. Predictably, the Soviets would not accept his warnings because ‘direct evidence was not forthcoming’. Depressed, he went back to Bletchley Park in February 1943, never to return to Moscow. He joined the staff at Bletchley Park and tried to keep the relationship going at a distance, ‘but the temperature was falling’. The Director, Commander Edward Travis, was only willing to allow the relationship to continue ‘if it is a solid gain for us’. The Polyarnoe naval listening station continued to function, but with the Soviets turning the tide on the Eastern Front they seemed to feel no need for further cooperation, and other contacts ‘petered out’.24 On 9 February 1944, London discussed the possibility of a visit to Britain by Soviet cypher experts and decided against it.25

Bletchley Park’s heated debate on what information to give to the Soviets was academic. All along, one of the KGB’s top agents, John Cairncross, had been working at Bletchley. Although Cairncross studied at Cambridge in the early 1930s, he was not recruited by Anthony Blunt, one of the key KGB talent scouts there, who found him both quarrelsome and arrogant. Instead, after Cairncross joined the Foreign Office in 1936, he was persuaded to work for Soviet intelligence by James Klugman, a prominent British Communist, who later served in the wartime Special Operations Executive. Although Cairncross was fearsomely intelligent, his difficult personality ensured that he was always being moved on. At the outbreak of the war with Germany he was sent to the Cabinet Office to work for the Cabinet Secretary, Lord Hankey. There he saw some of the early British thinking on the development of the atomic bomb. In 1941 he was moved to Bletchley Park, labouring in Hut Three on the Luftwaffe order of battle. His moment of triumph came in early 1943 when he was able to warn his KGB controller of the impending German armoured offensive at Kursk. Code-named ‘Operation Citadel’, this was the last great German push on the Eastern Front. It proved to be the largest tank battle of the Second World War, and the information provided by Cairncross proved to be important in launching an early attack upon the German tactical air force, much of which was destroyed on the ground. Stalin later awarded him the Order of the Red Banner in recognition of his achievement.26

Soon after Kursk, Cairncross moved again. He now returned to London and ended up in Section V, the counter-intelligence section of SIS, working alongside Kim Philby. Although he worked with Philby, Guy Burgess and indeed Donald Maclean, Cairncross was unaware of their common allegiance to Moscow, and believed he was the sole high-grade KGB agent in Whitehall. Bizarrely, he was caught in 1951 because of an official note in his handwriting found in the flat of Guy Burgess after Burgess had fled to Moscow with Maclean. Cairncross had given Burgess this quite innocently in the course of official business, without knowing he was a fellow spy. Once the note was found, Cairncross was followed, and MI5 surveillance believed they had caught him trying to meet with his KGB controller. Without hard evidence he could not be prosecuted, and he was merely asked to resign. Ironically, the Ultra material that Cairncross passed to the KGB was taken more seriously by Moscow precisely because it was stolen. Had the British handed it willingly to their ally, Stalin’s suspicious mind would almost certainly have devalued it.27

Cairncross was not the only KGB agent with access to Ultra. In late 1942, Anthony Blunt, another high-grade Soviet agent, was designated one of the two MI5 liaison officers who worked closely with Bletchley Park.28 Anxiety about KGB agents and subversion was yet another reason that the British kept working on Soviet traffic. Monitoring stations, notably the Metropolitan Police intercept station at Denmark Hill in south London, reported an upswing in traffic between Moscow and secret agents in Britain. There was also a British field unit, called the Radio Security Service, that hunted for illegal agent radio transmissions, and it told the same story, although the agent traffic could not be broken.29 John Croft, who worked at the GC&CS diplomatic code-breaking centre at Berkeley Street in London, was one of those who soldiered on with Soviet material. Croft was engaged on wartime Comintern traffic in Europe, known as ‘Iscot’, which could be read. Although circulated only to a very select group of individuals within Whitehall, this material mostly revealed a dutiful Soviet struggle against their shared enemy, Nazi Germany. There is no indication that this material was exchanged with Washington.30

Early British cooperation with the American code-breakers was also tentative. Again, the obstacle was obsessive security. Security problems existed on several different levels. The British and the Americans had cooperated on sigint during the First World War, but this had bequeathed a legacy of doubt and anxiety, even distrust. In November 1940, when reviving sigint cooperation with the Americans was discussed, Alastair Denniston was quick to point out that after the First World War the ‘notorious’ American code-breaker Herbert O. Yardley had published a tell-all book about his experiences. The very name ‘Yardley’ caused a shudder in British code-breaking circles. Yardley was now working for the Canadians, and GC&CS insisted that they sack him summarily before they were allowed to join the wartime sigint club.31 Indeed, the Canadians were told that other agencies ‘would not touch Yardley with a ten foot pole’.32

The British, and especially Sir Stewart Menzies, the Chief of SIS, were frosty towards the Americans, and regarded them as fundamentally insecure. By contrast, the Americans generously opted to share the secret of their spectacular code-breaking success against the Japanese ‘Magic’ diplomatic cypher with the British as early as January 1941, even handing over precious examples of their copies of the Japanese cypher machine. The British were ‘flabbergasted’. They did not expect the Americans to ‘simply walk in and plonk down their most secret cryptanalytical machine’.33 Yet the British remained reticent, and did not initially reciprocate fully with their knowledge of Enigma. The jibe about American insecurity had a certain irony, since the British chose to send one of the priceless American copies of the ‘Purple’ machine out to their naval base at Singapore shortly before it fell to the Japanese. The machine was delivered by ship just as the Japanese invasion of Malaya began, and disappeared into the chaos of battle. To this day its fate is unknown.34

Collaboration with Washington was also hard because American sigint was a house divided. Although William Friedman, the US Army’s best cryptologist, was busy advocating sigint cooperation with the British in early 1940, the US Navy’s chief code-breaker, Commander Laurance Safford, was adamantly set against working with allies. But after pressure from President Franklin D. Roosevelt the Navy had been won round, and the Americans sent a team of technical experts to Britain in early 1941. Known as the ‘Sinkov Mission’, they spent several weeks touring Bletchley Park and visiting outlying intercept stations. The British were willing to receive them because they knew the main focus of American attention was Japan. At this stage the British were keen to keep discussions focused on Japan, because this allowed them to hide the extent of their knowledge of the German Enigma system. Both Sir Stewart Menzies and Sir Alexander Cadogan were adamant that the Ultra secret would not be shared with the Americans.35 Laurance Safford later represented the first Anglo-American exchanges of late 1940 and early 1941 as a one-way street in which the Americans handed over their precious ‘Magic’ material on Japan but got nothing in return. In fact this is far from the case. Prescott Currier, one of the Americans who came to Bletchley in early 1941, recalled: ‘All of us were permitted to come and go freely and to visit and talk with anyone in any area that interested us.’36 Later that year, a select circle of American code-breakers were also given more details about Enigma.37

The hottest issue was the distribution of sigint to the policy-makers. In late May 1941, Brigadier Raymond Lee, the American Military Attaché in London, conveyed an American request for comprehensive intelligence exchange in the Far East. There followed painfully slow and complex discussions about who would get sigint with what levels of security: ‘The whole thing has been so tangled up,’ he complained.38 Sigint was also very confused in Washington. Unlike Britain’s GC&CS, American signals intelligence was less centrally organised, resulting in great rivalry between the armed services.39 Because the American wartime sigint organisation was divided between the Army and the Navy, one of the great problems for the British was cooperating with one without upsetting the other. Famously, the Americans solved the tussle over who would decrypt Japanese codes by agreeing that the Army would decode the material on the even days of the month, and the Navy on the odd days. A more ludicrous system for the division of labour would have been hard to devise.40

GC&CS might have been more centralised than the Americans, but it had less money. Expanded cooperation with America on Japan allowed GC&CS to shed some difficult code-breaking tasks. High-grade Japanese Army cyphers had proved impenetrable for a decade. By 1941, Bletchley Park was too busy with the European war, while its Far Eastern code-breakers were struggling to cope with the mass of material on Japanese espionage derived from low-level consular intercepts in South-East Asia. On 22 August 1941, Anglo-American cooperation lifted this task from their shoulders. During talks in Washington, Alastair Denniston persuaded the US Army that it should ‘take over investigation of Japanese main army cipher soon as priority commitment’. Shortly after, Captain Geoffrey Stevens from Singapore travelled to Washington carrying all the British material on the Japanese main army cypher.41 The British were glad to see the back of it. At the end of the war approximately 2,500 Americans would still be working on this one Japanese cypher to no avail.

All the while, Britain was also decyphering some American traffic. Amongst the decrypts selected for the personal perusal of Winston Churchill were those of many Allied and neutral countries. GC&CS was clearly working successfully on the American diplomatic code ‘Grey’ until December 1941.42 Remarkably, there was no embarrassment about this. In June 1941, while discussing comprehensive sharing of Far East intelligence, the British asked the US Military Attaché, General Raymond Lee, for his opinion on the security of American cyphers. This was the conduit through which sigint would pass between London and Washington. Lee replied tartly that the GC&CS already knew a great deal about this matter. He recorded in his diary:

The talk then turned again on the question of security. They wanted to know whether my despatches went by radio or cable and were relieved to hear that they went by cable, and were further relieved to hear that we have a direct wire straight into the War Department. However, I pointed out that this wire was subject to interception by their people here in England [GC&CS] and I had no doubt they had taken our messages and attempted to decipher them.

He added that it was now very much in the interests of GC&CS to be honest about the security of American cypher systems, ‘because the stuff that is going over it is more vital to them than to us’. Lee’s frank exchange with the British underlines one of the hidden benefits of cooperation between the Allied code-breakers. Once they began to share their most precious assets, ‘Magic’ and then eventually Ultra, improved communications security became paramount. London and Washington now had a vested interest in the impenetrability of each other’s messages. After all, if GC&CS could break American codes, then so, perhaps, could the Germans.43

Churchill eventually wrote to Roosevelt and owned up to British work on American diplomatic codes. ‘From the moment we became allies,’ he explained, ‘I gave instructions that this work should cease. However, danger of our enemies having achieved a measure of success cannot, I am advised, be dismissed.’ In fact, it is unlikely that all work on American traffic ceased. In areas such as the Middle East, Britain had a considerable incentive to continue to work on American commercial traffic, much of which was in commercial code or plain text. Indeed, a close reading of Churchill’s assurance to Roosevelt suggests that it might have related to diplomatic traffic only.44 Some GC&CS staff recall work on the traffic of American commercial attachés throughout the war, although as yet no documents have been released.45 Predictably, clear traffic from American oil companies was being intercepted in 1944 as they began to look for new markets in Europe.46

For this very reason, the US Army and Navy were agreed that nothing should be passed to the British about American code-making procedures, such as the Sigaba cypher machine. General George Marshall, the US Army Chief of Staff, specifically forbade any such exchange in September 1940.47 Anxiety about protecting national cypher systems persisted through the war on both sides of the Atlantic. In February 1945 Britain’s newly formed Cypher Policy Board debated a proposal by its Secretary, Captain Edmund Wilson, for ‘free and complete interchange’ with the Americans on cypher machine development, together with scrambler phones and secure speech.48 This horrified both Edward Bridges, the Cabinet Secretary, and Sir Stewart Menzies, Chief of SIS, and the idea was rejected. Cooperation on communications security would focus on machines specially designed for combined use.49

The gradual collapse of the British monopoly over Ultra intelligence paved the way for closer Anglo-American sigint cooperation. As we have seen, Bletchley’s initial idea for wartime cooperation was that the Americans would continue their pre-war focus on Japanese traffic; meanwhile the British would handle the work on Enigma, dispensing its product to the Americans as they saw fit. Although they had informed the Americans about Enigma in 1941, some precise details of processing had been withheld. Bletchley was determined to prevent the Americans working on Enigma in parallel, even though the Battle of the Atlantic gave Washington a legitimate need for Ultra intelligence. However, once the German Navy introduced an improved Enigma machine with four rotors, the British could not produce enough ‘bombes’ to deal with the increased number of tests required to break it.50 In September 1942, Joseph Wenger, who led the US Navy code-breakers, proposed spending $2 million to acquire no fewer than 230 four-wheel ‘bombes’. This was ten times the number available to Bletchley. John Tiltman, Britain’s Soviet code specialist, realised that American sigint was beginning to operate on an industrial scale, and that for Bletchley Park the game of ‘Ultra monopoly’ was surely up.51

In September 1942, Edward Travis and the head of Bletchley Park’s Naval Section, Frank Birch, travelled to Washington and concluded the ‘Holden Agreement’, which established full and integrated collaboration on German naval traffic, including Enigma. This was a key part of the emerging Anglo-American sigint relationship, and a constituent part of the secret alliance which still exists to this day.52 Travis’s hand was strengthened by the remarkable fact that the US Navy breathed not a word about the Holden Agreement to the US Army. The British therefore persisted in their hopes of keeping control over the processing of Ultra material derived from Luftwaffe and German Army traffic. Nigel de Grey, the Deputy Director of Bletchley Park, was apoplectic at the possibility of the Americans being allowed to duplicate further British work on Enigma. However, a US Army code-breaker based at Bletchley, Colonel Telford Taylor, suggested a tactful way forward. He advised his superiors in Washington that all they needed for the time being was a small ‘foothold’ in the work on Enigma, which would allow them to gain experience. More level-headed organisational types at Bletchley Park, such as Gordon Welchman, could see that the ability of the Americans to procure unlimited numbers of bombes was crucial, adding, ‘We certainly need help.’53 The result was the BRUSA agreement, a further crucial landmark in the construction of the Anglo-American sigint relationship. On 17 May 1943, Bletchley agreed to American participation in work on German Army and Air Force traffic. A second Holden Agreement on naval sigint followed in 1944. These treaties were of enormous importance, and paved the way for more ambitious post-war sigint alliances.

The exigencies of war had broken Britain’s cryptographic monopoly on Ultra. However, Ultra was a military system, representing the core work of Bletchley Park. There is no evidence that Britain and the United States concluded an overarching treaty on diplomatic or commercial sigint, the material that GC&CS worked on at Berkeley Street. In 1942, Alastair Denniston, who had been moved sideways to manage diplomatic sigint, arranged for cooperation on a number of specific countries such as Italy, France, Spain, Portugal, Japan and, of course, Germany. However, this was done on an ad hoc basis. There was no diplomatic BRUSA agreement. It seems that the Americans were not intercepting and working on a range of materials that would have prompted a wider deal. Typically, Denniston told John Tiltman, with evident relief, ‘They do no work on any of the Near Eastern governments.’54

Denniston’s main point of contact in the United States was William Friedman and the US Army code-breakers, who dominated American work on diplomatic systems. The Americans were keen to cooperate, since up until 1941 the US Army had been intensely focused on the diplomatic cyphers of Japan. In 1940 the Americans lost access to Japan’s diplomatic cypher, and it was only recovered as the result of a prodigious effort by a team under Frank Rowlett. By contrast, British code-breakers were working on the diplomatic cyphers of some twenty-six different countries.55 Therefore, when the Americans offered access to ‘Magic’, the British reciprocated with a wide range of diplomatic material, including high-grade Italian systems. Then, in March 1942, John Tiltman visited Washington and brought with him Spanish and Vichy French cyphers. Given the arrival of American forces in the Mediterranean, this was valuable material. By 1944 the Americans had received more material from the British on diplomatic cyphers used by the Greeks, Hungarians, Iranians and Iraqis. However, the processing went on behind a curtain. Denniston asked at one point, ‘Do they actually work on the stuff which we send them, or do they simply put it in their library?’ Diplomatic cyphers from countries that the British considered to be client states, such as Egypt, were withheld.56

Sharing diplomatic product caused some embarrassing problems. Foreign diplomats in London or Washington often reported their conversations with British officials in the messages they sent home. The British sometimes did not want the Americans to ‘listen in’ on these conversations, since they might involve ‘disparaging remarks about American policy or officials’. Therefore, they developed a special reserved series called ‘Res’,

that contained material that was not to be given to the Americans. This was not an effective solution, because, as Alexander Cadogan, the senior official at the Foreign Office, explained to Stewart Menzies, the Americans would often obtain and break some of the same traffic themselves, and so would ‘become suspicious’. By the spring of 1944 the Americans clearly knew about ‘Res’, and pressed the British to abandon the practice. However, Cadogan refused, since the war was drawing to an end, and the antagonistic politics of post-war settlements were looming.57

The Americans nurtured their own anxieties. Would Anglo-American sigint cooperation continue after the war? As early as 1942, Colonel Alfred McCormack, one of the more important visitors to Bletchley, warned his superiors in Washington that the British were ‘very realistic people’, and so would ‘certainly at some time – possibly while the war is still on – resume work on United States communications’.58 However, continued convergence of Anglo-American sigint was ensured by early fears of the Soviet Union, which were visible as early as 1942. Senior officers on both sides of the Atlantic, including Field Marshal Lord Alanbrooke and General Douglas MacArthur, were of one mind on the ‘Russia problem’. On 31 July 1942, Geoffrey Stevens, a code-breaker from GC&CS, went out to Arlington Hall, the US Army’s code-breakers’ centre in Washington. One of the subjects he discussed there was the Soviet Union, and he was fascinated to learn that the Americans were intercepting all the Soviet traffic in and out of Washington. They were also collecting Soviet traffic elsewhere, for example between Moscow and the Soviet Embassy in Tokyo. He reported that the Americans ‘do nothing about it at the moment’ by way of decryption, since they were so pressed for code-breaking capacity against the Axis. However, sooner or later, he added, ‘They will inevitably try and break this since they do not trust the Soviets further than they could throw a steam-roller.’59 Much as Stevens predicted, the Americans began a Soviet Group in February 1943. Meanwhile, the British moved their own existing Soviet team from Ryder Street in London to larger premises at Sloane Square in late 1944.60 Although the two allies were still working in isolation on the ‘Russia problem’, the foundation of future collaboration was already emerging.

Anxiety about the Soviet Union increased markedly during early 1944. By April the Red Army was pushing into eastern Hungary, and this filled Moscow with a newfound confidence. Stalin’s determination to impose a Communist government on Poland was already evident, and pointed to future trouble. Some British diplomats in the Foreign Office remained hopeful about the possibility of post-war cooperation with the Soviet Union, but their military colleagues did not share their optimism. Indeed, the main future strategic planning body in Whitehall, the Post Hostilities Planning Committee, which was shared between the diplomats and the military, tore itself apart over this issue. The Foreign Secretary Anthony Eden had to step in in late 1944, and banned the further circulation of its papers. One staff officer lamented that there were to be ‘no more games of Russian scandal’. Russia was now a forbidden subject, and between late 1944 and early 1946 Britain’s main body of intelligence analysts, the Joint Intelligence Committee (JIC), did everything it could to avoid discussing the dreaded subject of the Soviet Union.61

Accordingly, it was only in June 1945 that the American code-breakers formally proposed to the British that they cooperate against the Soviet Union, giving the overall programme the code name ‘Bourbon’. The formal Anglo-American collaboration on the wider ‘Russian problem’ was so incredibly secret that it was not written down, and amounted to a simple handshake between Group Captain Eric Jones, the British sigint liaison officer in Washington, and a senior American naval officer in June 1945. Meanwhile, all eyes were on the Allied reoccupation of Europe and the remarkable sigint prizes that were even now being recovered from the smouldering ruins of the Third Reich.62

3 Every War Must Have an End

On 26th August one of the [German] operators from Army Group, South Ukraine…suddenly broke into violent remarks about Hitler, using the peculiarly foul language in which the Germans delight. The operator at Supreme Army Command tried to shut him up in equally filthy language. This interchange lasted for about ten minutes…

The incident is only noteworthy as a possible indication of the way things are going.

Nigel de Grey, Deputy Director at Bletchley Park, to Sir Stewart Menzies, 14 September 19441

By the autumn of 1944 the Second World War was ending and the Cold War had, to all intents and purposes, already begun. In the east, the German Army was collapsing fast, and by September Soviet forces were at the borders of Prussia. A month later, American forces had entered Germany from the west, capturing the ancient town of Aachen. While much bitter fighting lay ahead, the minds of officials in London, Washington and Moscow were increasingly focused on the post-war settlement. Wartime relations with Russia had never been easy. Stalin’s intense and unwavering suspicion was underlined by the fact that, throughout the war, he had refused to leave Soviet-controlled territory to meet Churchill and Roosevelt. Harsh Soviet behaviour in newly occupied areas like Poland already pointed to post-war confrontation and rivalry, and all eyes were on the advance into Germany.

Britain and the United States were gearing up for piratical raids on the headquarters and laboratories of a collapsing Third Reich, and Axis sigint material was the treasure that was most actively sought. A joint Anglo-American planning group began consulting with Bletchley Park about what material it wished to scoop from an occupied Germany. By early 1945, Intelligence Assault Units were moving into Germany alongside the fighting elements of Allied formations, looking for all kinds of top-secret German experimental weapons. Bletchley Park despatched its own Target Intelligence Committee teams, known as ‘TICOM teams’, made up of a mixture of British and American personnel, to seek out cryptographic equipment and sigint personnel from Germany. The whole TICOM programme was run on what Commander Edward Travis called ‘an entirely inter-allied’ basis.2

Suddenly, boffins in glasses and cardigans found themselves turned into amateur commandos. Whisked away to a quarry near Bletchley, those selected for this task were given a short course in the use of sub-machine guns and hand grenades. They began on the Thompson sub-machine gun, but soon found the lighter Sten gun to be an easier weapon to handle. None of them performed well, but nevertheless they were soon on their way to Hitler’s ‘Alpine Lair’ at Berchtesgaden. Major Edward Rushworth, one of the senior British officers from Hut Three, led a TICOM team of a dozen officers, accompanied by Selmer Norland, an American stationed at Bletchley Park. They arrived at the major German headquarters at Augsburg on 8 May, VE-Day. Augsburg had been home to the famous German ‘Fish’, or Geheimschreiber, the encyphered teleprinter which Bletchley had eventually defeated with the mighty ‘Colossus’ computer. Sadly, all these beautiful machines, lovingly manufactured by Lorenz, had been smashed and the cypher wheels had gone. The dejected team surveyed the debris. However, a day later their spirits rebounded when they gleefully recovered a single intact late-model ‘Fish’ from a town on the Austrian border.3

On 12 May 1945 they reached Hitler’s Alpine retreat. The Führer’s accommodation had been heavily bombed, but a hundred feet below ground was a maze of bunkers and tunnels to explore, including an emergency power station and a complete telephone exchange. No more cypher machines seemed to be in evidence, and the mission was tailing off when, as a last task, Rushworth set off for nearby Rosenheim on the Austrian border, to question a cryptographer who had been working for the German High Command (OKW). While they were there, a group of other German prisoners sent a message asking to speak to the ‘proper people’. This team had served in the OKW headquarters sigint units and now revealed that, terrified of the rapid Soviet advance, they had buried their equipment under the pavement in front of their headquarters. Called ‘OKW-Chi’, they had successfully broken what was referred to as ‘Russian Fish’. This was an encrypted Soviet military teleprinter that achieved an early version of packet switching, breaking each message into nine different parts and routing it along separate channels, before reassembling it. The Germans had already worked out that their code-breaking triumph would have post-war value, and hoped to sell themselves on as a complete team.4

They were not disappointed. By 23 May they had been encouraged to unearth and set up their equipment, allowing them to resume decrypting Soviet command traffic. The Bletchley team were in awe of this vast technical display, which was eventually packaged up again in over a hundred boxes and chests. The eight tons of equipment and the complete German staff were loaded onto five lorries, which then wound their way slowly through a devastated Germany towards Bletchley. They arrived on 6 June 1945, and the equipment was set up and tested at the nearby radio station of Wavendon Manor.5 The German team was later employed intercepting Soviet encyphered teleprinter traffic which the British code-named ‘Caviar’, and although the messages were mostly about administration rather than policy or strategy, they provided rare insights into the daily activities of Soviet armed forces in post-war Europe.6 More treasures followed, and ultimately a further five tons of documents pertaining to Soviet codes and cyphers would arrive. In mid-June, Edward Travis asked Russell Dudley-Smith, a senior Bletchley Park officer, to try to establish some priority in exploiting the mountain of material now pouring in, but little did they know that they would still be working on this material in 1951.7 One-of-a-kind equipment stayed in Britain, while any duplicates were shipped to America.8

Yet another important haul was brought in by Colonel Paul Neff, an American who headed TICOM Team 6. This group included William Bundy, later US Assistant Secretary of State under President Lyndon B. Johnson, and Geoffrey Stevens from Bletchley Park. In April 1945 they pushed into southern Germany at Magdeburg, near Leipzig, and took control of a castle at Burgscheidungen which had recently been the headquarters of a code-breaking unit of the German Foreign Ministry called the Balkanabteilung, whose tasks had largely focused on Soviet and Balkan traffic. The fourteen staff and their documents were flown to Britain and taken to Bletchley Park. Burgscheidungen was in an area that would later be designated as part of the Soviet Zone, so Neff destroyed all traces of the German code-breakers’ presence before departing.9

The British caught Generalmajor Klemme, the Senior Commander of Radio Intelligence for the Luftwaffe, at the Husum-Milstedt intercept station on 19 May 1945. At first he was taken to Neumuenster Prison, but from there he was brought to Britain, and worked with the Allies on sigint in Germany until 10 March 1948, when he was considered to have been drained of all he knew about Soviet communications. On 1 May 1945, Major Oeljeschaeger and Major Beulmann from the Berlin Cryptographic Centre, which had been based in a stable block of the Marstall-Neues Palais at Potsdam, had fled in the direction of Hitler’s complex at Berchtesgaden. A few days later, with the Allies closing in, they stopped at Viehoff to burn all the records of Branch 3, and they fell into Allied hands on 22 May near Munich. On 5 July they were flown to Britain and placed in a special camp. They were surprised to be welcomed by their Branch Chief, Lt Colonel Friedrich, who had been captured before them. By June 1945 the British and Americans had scooped up most of the senior Luftwaffe sigint officers whose traffic they had listened to assiduously for much of the war.10

The TICOM teams were competing with the Soviets, who were also swooping on German cryptographic assets. To their surprise, Bletchley Park discovered that the Soviets had taken over some German Enigma-based communications nets and Fish teleprinters, and had begun using them for their own purposes. However, initial hopes of a post-war dividend from the breaking of these machines were quickly dashed. Roy Jenkins, who was then working at Bletchley Park, recalls this odd interlude in May 1945:

When the Russians got to Berlin they took over the Fish machines in the War Ministry, somewhat changed the settings, and proceeded to use them for sending signals traffic to Belgrade and other capitals in their new empire. We continued to do the intercepts and played around with trying to break the messages. We never succeeded. I think it was a combination of the new settings being more secure (which raises the question of how much the Russians had found out about our previous success) and the edge of tension having gone off our effort.

Elsewhere, Allied recovery teams regularly overran German sigint operations that were still chattering away, producing decrypts of mid-level Soviet Army Group traffic.11

The timing of raids on German sigint centres was a precarious matter. If they were captured too early there was a risk that this would cut off a flow of valuable material that Bletchley was intercepting, or else would alert the Germans to the fact that the British knew more about their cyphers than was desirable. London was especially anxious to avoid freelance raiding activities that might be counter-productive. As early as May 1944 the London Signals Intelligence Board, the supreme governing board which met monthly to set overall British sigint policy, learned that some independently-minded British intelligence officers in the Middle East were planning to use the Special Operations Executive to raid enemy signals intelligence centres in the Balkans. Sir Stewart Menzies, who chaired the board, warned them sternly that operations against such centres were ‘highly undesirable’, and that action should ‘on no account be undertaken’ without prior personal authority from him.’12

Bletchley’s corporate takeover of the Axis sigint effort was not limited to Germany. There were even greater TICOM dividends in occupied Italy.13 Many countries competed for the services of the talented Italian cryptanalysts. After the Italian surrender in 1943, some eighty Italian code-breakers under Major Barbieri continued to work for the Germans at a station near Brescia in northern Italy. At the end of the war in Europe they were at last interrogated in Rome, and proved to have a large quantity of material, including photocopies of the codebooks of Turkey, Romania, Ecuador and Bolivia. They had also reconstructed some of the codebooks from France, Switzerland and the Vatican, and had smaller amounts of British and American traffic. During the spring of 1945 Barbieri’s unit had been concentrating on French diplomatic traffic, ‘a large number being messages to Paris either from Bonnet [French Ambassador] in New York or from Catroux [French Ambassador] in Moscow’. This traffic offered insights into subjects as diverse as Soviet-Yugoslav relations, Soviet policy in Germany, French economic negotiations with the United States and French plans for exploiting the Saar coal mines in Germany.14

With British encouragement, this precocious Italian unit worked on into the post-war period, without deviating from its French target. The diplomatic unit at Berkeley Street was already doing extensive work on Britain’s European allies, regarding them as either insecure or untrustworthy, or both. Much of this suspicion stemmed from a sense of indignation at their behaviour in 1940. In November 1944, Churchill wrote to Eden: ‘The Belgians are extremely weak, and their behaviour before the war was shocking. The Dutch were entirely selfish and fought only when attacked, and then for a few hours…’ General de Gaulle’s Free French government in exile, as other historians have shown, came in for especially close attention from the code-breakers during the war, and this continued into 1946.

During the important diplomatic conferences that marked the end of the war, Jimmy Byrnes, the new American Secretary of State, was apparently more eager to see decrypted French material than anything else, concerned that Paris was likely to be working with Moscow.15 French traffic from Moscow was of great interest to London because the former French Air Minister, Pierre Cot, had indeed begun a special diplomatic mission to Moscow to examine the possibility of cooperating against Germany in post-war Europe.16

French traffic provided the British and Americans with a fabulous window on the diplomacy of Western Europe.17 Indeed, in mid-1946 half the US Army code-breakers’ end product was based on intercepting French communications.18 Alarmingly, the French still seemed keen to develop a close relationship with Stalin.19 The traffic from French Embassies in Eastern Europe proved especially interesting. Typically, an intercept from the French Embassy in Tirana gave detailed information on the balance of power in the Albanian Cabinet and the waning power of the pro-Moscow elements, and intercepted French intelligence traffic sometimes offered information about the KGB.20 With the work on Soviet codes still gaining momentum, the chatter of other countries that were talking to Moscow provided insights into their thinking. On 13 August 1945, Edward Travis sent Joseph Wenger, the senior American naval code-breaker, a long missive about cooperation on post-war French and Dutch systems, and explained British plans ‘to increase the effort here, especially on French’, adding that British plans to focus on Paris ‘are going into effect at an early date’. French, Spanish, Portuguese and Latin American traffic was soon consolidated into a single group under Josh Cooper.21

Major Barbieri was proud of the work of his Italian code-breakers against the French, but he pressed for more staff. So many of the best cryptographers, he complained, had been captured by the French in North Africa, adding, ‘the French are now employing them in their own service!’ Nevertheless, the British concluded that the Italians were ‘doing remarkably well with the limited reserves at their disposal’.22 By mid-1946 they were giving them new tasks, including Soviet traffic which came from military cypher machines at division level code-named ‘Taper’. British liaison officers with the Italians were working closely with code-breakers in Britain on the identification of new Taper groups. Senior Italian sigint officers knew that Taper traffic ‘which had been taken with so much depth and continuity for the past month’ was Soviet in origin, but many of their underlings were in a state of blissful ignorance about what they were collecting and who the ultimate customer was.23

The efforts of TICOM were not exclusively directed towards raiding priceless sigint secrets from the Germans, the Italians and the Japanese. They were also concerned with protecting Britain’s own secret communications. Until late 1943, Bletchley Park regarded weak security as a problem restricted to Britain’s allies. But the ability to read German messages had revealed a number of unexpected security nightmares for the Allies. Ultra had shown Britain’s code-breakers that the Germans could read many of the codes of the Allies, such as those of the Soviets and the Free French. In Asia, terrible cypher security and serious human agent penetration ensured that Chinese codes were effectively an open book to the Japanese, even though Tokyo’s code-breakers were mediocre. Accordingly, keeping Britain’s secrets safe meant keeping them away from many of her allies, whose communications were being read by friend and foe alike.24

By the autumn of 1943 the security situation looked much worse. The Italians had now capitulated, and captured Italian code-breakers revealed their successes against British codes. Captain Edmund Wilson, who helped to look after cypher security at Bletchley Park, held prolonged ‘conversations’ with Commander Cianchi, head of the Italian Cryptographic Bureau in Rome, and his staff during late 1943. Wilson explained that he could hardly call them ‘interrogations’, since Cianchi had given all of Italy’s secret information so happily and freely. Wilson said that ‘very valuable information’ on the breaking of British naval cyphers had been obtained, and that Britain was ‘extremely fortunate’ to have the cooperation of its former opponents. He pressed his colleagues to be ‘very careful indeed in the use they made of the information’ from these sources.25

Pulsuz fraqment bitdi.

18,34 ₼
Yaş həddi:
0+
Litresdə buraxılış tarixi:
28 dekabr 2018
Həcm:
902 səh. 4 illustrasiyalar
ISBN:
9780007357123
Müəllif hüququ sahibi:
HarperCollins

Oxşar kitablar