A 1998 U.S. Department of Commerce report provided the following assessment on the emergence of the internet:

"The internet's pace of adoption eclipses all other technologies that preceded it. Radio was in existence 38 years before 50 million people tuned in; TV took 13 years to reach that benchmark. Sixteen years after the first PC kit came out, 50 million people were using one. Once it was opened to the general public, the Internet crossed that line in four years".

Here, Felix Debieux returns to the site and considers the military origin of the internet - and the role of the Vietnam War in that.

A DEC PDP-10 computer. Source: Gah4, available here.

Despite the hold that the internet now has on everyday life, it is difficult to imagine it as something that was ever invented. Unlike the car, the telephone, or the aeroplane, the internet by comparison has managed to achieve ubiquity without us being able to point to an obvious creator. Of course, we see traces of the internet in apps, video games and email, but there is nothing we can picture holding, touching, or turning over in our hands for inspection.

Maybe it is the ubiquity of the internet which makes it such a daunting prospect for historians. With so many applications, how would one even begin to trace its origins? Historians might also be put off by the technical workings of the internet – the circuit boards, networks and switches. The truth, however, is that the history of the internet is more straightforward than we might expect. In fact, if it was not for its simplicity, it is arguable that the meteoric success of the internet would never have occurred.

The emergence of the internet is a story of global proportions. Its inventors (yes, there were inventors!) include the French government-sponsored computer network Cyclades, England’s National Physical Laboratory, Xerox, and the University of Hawaii. Normally placed at the foreground of the typical story are the freewheeling creatives and plucky entrepreneurs of Silicon Valley. Here, the internet is typically cast as a great liberating force that helped to decentralise power and spread democracy around the globe.

The glitz of this conventional narrative, however, obscures a much more sinister history explored in great detail by investigative reporter Yasha Levine. In Surveillance Valley: The Secret Military History of the Internet, Levine shifts the focus of the story onto one of its most important and yet consistently overlooked characters: The Advanced Research Projects Agency (ARPA). A generously funded research arm of the U.S. Department of Defense, ARPA was born out of America’s insecurities during the Cold War. Its remit grew quickly, however, to encompass a wide array of counterinsurgency and surveillance projects which played a key part in the genesis of the internet.

The best starting point for this story is the Cold War, when ARPA first appears on the scene.

The Cold War and ARPA

The origins of the internet are rooted in the heightened international tensions of the Cold War, a period during which the U.S. and Soviet Union vied for technological supremacy. Each boasted a deadly arsenal of nuclear weapons, and populations on both sides lived in fear of surprise attack. In the U.S., paranoia peaked in 1957 with the launch of the Soviet satellite Sputnik 1. The success of the Soviet’s in reaching space first shattered America’s sense of exceptionalism. Politicians seized on the launch as a sign of U.S. military and technological weakness. How had America fallen behind the reds in something so vital?

President Dwight Eisenhower was vilified for appearing to have fallen asleep at the wheel. Generals and political rivals spun tales of an impending Soviet conquest of Earth and space, and pushed for greater military spending. Even Vice President Richard Nixon publicly criticised Eisenhower, informing business leaders that the technology gap between America and the Soviet Union was too great for them to expect a tax cut. As the public reeled from defeat in the so-called Space Race, Eisenhower knew that the only way to save face was to do something big, bold and very public.

It was against this backdrop that Eisenhower established ARPA in 1958. The idea was straightforward. With a small staff count and a large budget, ARPA would function as a civilian-led unit housed by the Pentagon. It would neither build or run its own research facilities, but instead would operate as an executive management hub that identified priorities and then siphoned the research out to universities, private research institutes, and military contractors. ARPA would bring together some of the top scientific minds in the country, with the aim of keeping American military technology ahead of the communists. One area given priority were the perceived vulnerabilities in U.S. computer communications. Indeed, military commanders were very keen to develop a computer communications system without a central core, and with no obvious headquarters that could easily be knocked out by a single Soviet strike (thus crippling the entire U.S. network).

The ARPANET

ARPA was therefore tasked with testing the feasibility of a large-scale computer network. Lawrence Roberts, the first person to have succeeded in connecting two computers, was responsible for developing the network, and collaborated closely with scientist Leonard Kleinrock. By 1969, the first packet-switch network was developed and Kleinrock successfully used it to send messages to another site. The ARPA Network, or ARPANET - the grandfather of the internet as we know it - was born.

Originally, there were only four computers connected when the ARPANET was created. These were located in the computer labs of UCLA (Honeywell DDP-516 computer), Stanford Research Institute (SDS-940 computer), University of California, Santa Barbara (IBM 360/75) and the University of Utah (DEC PDP-10). The first data exchange over this new network was between computers at UCLA and the Stanford Research Institute. On their first attempt to log into Stanford's computer by typing "log win," UCLA researchers crashed their computer after typing the letter “g”.

In time the network would expand, and different models of computer were connected. This gave rise to inevitable compatibility issues. The solution rested on an improved set of protocols called TCP/IP (Transmission Control Protocol/Internet Protocol) that were designed in 1982. This worked by breaking data into IP (Internet Protocol) packets, akin to individually addressed digital envelopes. TCP (Transmission Control Protocol) then ensured that the packets were delivered from client to server and reassembled in the right sequence.

Through the ARPANET, we see the emergence of several computing innovations which we take for granted today. Notable examples include email (or electronic mail), a system that allows for simple messages to be sent to another person across the network (1971), telnet, a remote connection service for controlling a computer (1972) and file transfer protocol (FTP), which allows information to be sent from one computer to another in bulk (1973).

In its early days, the ARPANET was seen largely as a tool for academic engineers and computer scientists. It linked departments at several universities into a wider network, which by 1973 had expanded to over 30 intuitions in locations as far apart as Hawaii and Norway. While we could choose to end the story of the ARPANET here, it is only by following the roots deeper that we are able to acquire a fuller understanding of how the technology came to be. Indeed, while academic scientists were busy using the ARPANET to establish connections between research sites, it was a conflict taking place thousands of miles away which provided the perfect conditions for the technology to be tested and refined.

Vietnam

Some further context is likely to be useful here. During the Cold War, the U.S. faced regional insurgencies against allied governments. From Algiers to Laos, from Nicaragua to Lebanon, most of these conflicts shared common characteristics. They were born out of local movements, they recruited local fighters, and they were supported by local populations. No matter how many nuclear weapons the U.S. boasted, countering insurgencies of this nature was not something that a conventional military operation was equipped to deal with. There was an obvious need to modernise and expand U.S. military capabilities, the case for which was made very clear by President Kennedy in his 1961 message to Congress:

“The Free World’s security can be endangered not only by a nuclear attack, but also by being slowly nibbled away at the periphery, regardless of our strategic power, by forces of subversion, infiltration, intimidation, indirect or non-overt aggression, internal revolution, diplomatic blackmail, guerrilla warfare or a series of limited wars […] we need a greater ability to deal with guerrilla forces, insurrections, and subversion”.

In essence, Kennedy envisioned cleverer and more sophisticated ways of fighting communism. What better institution was there than ARPA to develop the modern technological capabilities which the U.S. so desperately needed? In response to Kennedy’s speech, the CIA, the Pentagon, and the State Department drew up plans for a massive programme of covert military, economic, and psychological warfare initiatives to deal with one of America’s key geopolitical problems: the growing insurrection in Vietnam.

Among the biggest beneficiaries in the funding pipeline was ARPA’s Project Agile, a high-tech counterinsurgency programme which aimed to support the government of South Vietnam in researching and developing new techniques for use against the Vietcong. More specifically, Project Agile sought to develop weapons and adapt counterinsurgency gadgets suited to the dense, sweltering jungles of South East Asia. Some of the initiatives in the project included:

§ Testing light combat arms for the South Vietnamese military, which led to the adoption of the AR-15 and M-16 as standard-issue rifles.

§ Developing a light surveillance aircraft that glided silently above the jungle canopy.

§ Formulating field rations and food suited to the hot, wet climate.

§ Developing sophisticated electronic surveillance systems and elaborate efforts to collect all manner of conflict-related intelligence.

§ Working to improve the function of military communication technology in dense rainforest.

§ Developing portable radar installations that could be floated up on a balloon.

Through Project Agile, ARPA would push the boundaries of what contemporaries considered technologically possible. It pioneered electronic surveillance systems that were decades ahead of their time.

Agile was by no means the only ARPA battlefield project. Indeed, among ARPA’s most ambitious initiatives was Project Igloo White, a multi-billion-dollar computerised surveillance barrier. Operated out of a secret air force base in Thailand, the project involved depositing thousands of radio-controlled seismic sensors, microphones, and heat and urine detectors in the jungle. These eavesdropping devices, disguised as sticks or plants and usually dropped from aeroplanes, transmitted signals to a centralised computer control centre which alerted technicians to any movement in the bush. If movement was detected, an air strike was called in and the area was blanketed with bombs and napalm. Igloo White could be described as a giant wireless alarm system that spanned hundreds of miles of jungle. As the US Air Force explained: “we are, in effect, bugging the battlefield”.

Soon we will see how ARPA’s burgeoning surveillance and counterinsurgency expertise fed into the development of the ARPANET. But first it is necessary to understand ARAP’s deepening role in the conflict.

Know thy enemy

While the development of high-tech counterinsurgency and surveillance equipment was an important dimension of ARPA’s role in the war, key to our story is the application of the technology to the examination and study of rebellious peoples and their culture. This was the truly visionary part of ARPA’s mandate: to use advanced science not just to develop weapons, but to beat the motivated and well-disciplined Vietnamese insurgents. The idea was to understand why they resisted, to weaken their resolve, to predict insurgency, and to prevent it from maturing. The value of this science to the U.S. military, deployed to a hostile environment which they did not understand, was obvious.

In a step up of its activities, ARPA was given license to figure out how to weaponize anthropology, psychology, and sociology in support of the military’s counterinsurgency work. ARPA doled out millions of dollars to studies of Vietnamese peasants, captured Vietcong fighters, and rebellious hill tribes of northern Thailand. Swarms of ARPA contractors—anthropologists, political scientists, linguists, and sociologists—put poor villages under the microscope. Their work involved measuring, gathering data, interviewing, studying, assessing, and reporting on their inhabitants. As Levine explains, “the intention was to understand the enemy, to know their hopes, their fears, their dreams, their social networks, and their relationships to power”.

Studies commissioned by ARPA contractors sought answers to the questions which nagged at the American psyche: why were North Vietnamese fighters not defecting to our side? What was so appealing about their cause? Don’t they want to live like we do in America? Why was their morale so high? 2,400 interviews of North Vietnamese prisoners and defectors were conducted, generating tens of thousands of pages of intelligence. Perhaps the clearest illustration of ARPA’s application of science to socio-cultural problems was its work on the Strategic Hamlet Programme. The programme was a pacification effort that had been developed as part of Project Agile, and involved the forced resettlement of South Vietnamese peasants from their traditional villages into new areas that were walled off and made “safe” from Vietcong infiltration.

In addition, ARPA funded several projects which sought to pinpoint the precise socio-cultural indicators that could be used to predict when tribes would go insurgent. One initiative involved a team of political scientists and anthropologists sent from UCLA and UC Berkeley to Thailand to map out the religious beliefs, value systems, group dynamics and civil-military relationships of Thai hill tribes, focusing in particular on predictive behaviour. A project report summarised the objective: “to determine the most likely sources of social conflict in north east Thailand, concentrating on those local problems and attitudes which could be exploited by the communists”.

Not content with merely gathering intelligence, ARPA also experimented with how it could shape indigenous populations. For example, one study carried out for ARPA by the CIA-connected American Institutes for Research (AIR) attempted to gauge the effectiveness of counterinsurgency techniques deployed against rebellious hill tribes. Techniques included assassinating tribal leaders, forcibly relocating villages, and using artificially-induced famine to pacify rebellious populations. A 1970 investigation for Ramparts magazine detailed the effects of these brutal counterinsurgency methods on the Meo hill tribe: “conditions in the Meo resettlement villages are harsh, strongly reminiscent of the American Indian reservations of the 19th century. The people lack sufficient rice and water, and corrupt local agents pocket the funds appropriated for the Meo in Bangkok.”

While ARPA’s activities in Vietnam were certainly disturbing, we have yet to explain their relevance to the creation of the ARPANET. An understanding of how ARPA’s science projects were adapted from the jungles of Vietnam and used back home on American citizens will bring us closer to an explanation.

Spying at home

To contemporaries, America in this period must have felt like it might explode at any second. Race riots, militant black activism, left-wing student movements, and anti-war protests were rife. In 1968 alone, Robert Kennedy and Martin Luther King Jr. were both assassinated, the death of the latter sparking riots across the country. In November the following year, three hundred thousand people descended on Washington D.C. for the largest anti-war protest in American history. Suspicion of communist agitation was high, and U.S. intelligence services focused their resources on weeding out communist troublemakers - both real and alleged.

This climate of suspicion and paranoia afforded ARPA the perfect conditions in which to refine everything it had learned in Vietnam in a domestic setting. Take its work on the Strategic Hamlet Programme. Perhaps the most unsettling aspect of the initiative is that it was, at least in part, always intended to serve as a model for counterinsurgency operations elsewhere in the world – including against black Americans living in inner cities back home. This was made explicit in the project proposal:

“The potential applicability of the findings in the United States will also receive special attention. In many of our key domestic programs, especially those directed at disadvantaged sub-cultures, the methodological problems are similar to those described in this proposal […] the application of the Thai findings at home constitutes a potentially most significant project contribution”.

Once the war in Vietnam had concluded, ARPA researchers returned to the U.S. and began applying the lessons of the programme to the domestic problems of racial and socio-economic inequality.

One of the strongest insights we have into ARPA’s domestic turn comes from a shocking exposé, which dragged the agency’s work out into the sunlight. From 2nd June 1975, NBC correspondent Ford Rowan appeared on the evening news every night for a week to warn millions of viewers that the country’s military had built a sophisticated computer communications network and was using it to spy on Americans:

“Our sources say, the Army’s information on thousands of American protesters has been given to the CIA, and some of it is in CIA computers now. We don’t know who gave the order to copy and keep the files. What we do know is that once the files are computerized, the Defense Department’s new technology makes it incredibly easy to move information from one computer to another […] this network links computers at the CIA, the Defense Intelligence Agency, the National Security Agency, more than 20 universities, and a dozen research centres, like the RAND Corporation”.

Rowan was no doubt talking about the ARPANET, and warning his fellow citizens of the threat that its networking capabilities posed to American freedoms. This was a story which Rowan had pieced together from the testimony of whistleblowing ARPA contractors, who had grown increasingly uncomfortable with this new application of the ARPANET. His reporting offers a vital window into how the U.S. defence and intelligence communities were using network technology to snoop on Americans in the very earliest version of the internet. Surveillance was baked into its original design.

A demonstration of the technology in action can be gleaned from its use against communists, both real and alleged, living in American.

Targeting communists

One of the most illustrative domestic counterinsurgency and surveillance operations conducted against Americans went by the name CONUS Intel (Continental United States Intelligence). Managed by U.S. Army Intelligence Command, CONUS Intel deployed thousands of undercover agents with the aim of infiltrating anti-war groups, monitoring left-wing activists, and filing reports in a centralised database on millions of U.S. citizens. The scale of CONUS Intel was far reaching. Agents reported on the smallest of protests, monitored labour strikes, and kept detailed notes on union supporters. At the 1968 Democratic National Convention they even tapped the phone of Senator Eugene McCarthy, a vocal critic of the Vietnam War. Agents infiltrated a meeting of Catholic priests who protested the church’s ban on birth control. They spied on the funeral of Martin Luther King, mingling with mourners and recording any discussions they heard. They even infiltrated the 1970 Earth Day festival, photographing and filing reports on anti-pollution activists.

In January 1970, former military intelligence officer and whistleblower Christopher Pyle published an exposé in the Washington Monthly which revealed details of CONUS Intel to the public. “When this program began in the summer of 1965, its purpose was to provide early warning of civil disorders which the Army might be called upon to quell in the summer of 1967,” reported Pyle. “Today, the Army maintains files on the membership, ideology, programs, and practices of virtually every activist political group in the country”. This is identical to the way in which ARPA had curated the lives, behaviours and cultures of Vietnamese insurgents.

Pyle’s exposé went on to describe how CONUS Intel’s surveillance data was encoded onto IBM (International Business Machines Corporation) punch cards and fed into a computer located at the Army Counter Intelligence Corps Centre at Fort Holabird. The centre was equipped with a terminal link that could be used to access almost a hundred different information categories and to print out reports on individual people. As Pyle explained:

“In this respect, the Army’s data bank promises to be unique. Unlike similar computers now in use at the FBI’s National Crime Information Center in Washington and New York State’s Identification and Intelligence System in Albany, it will not be restricted to the storage of case histories of persons arrested for, or convicted of, crimes. Rather it will specialize in files devoted exclusively to the descriptions of the lawful political activity of civilians”.

As with Rowan’s exposé, Americans were warned again that their freedoms were under threat. In this case, we see how ARPA’s techniques and technology - which had first been used to predict communist uprisings in Vietnam - were now being applied for precisely the same purpose against perceived domestic enemies. Indeed, the same ARPANET which academics had used to link research sites also provided the intelligence community with some of its earliest network surveillance capabilities. On a fundamental level, the academic and military applications were no different. Weeding out communists required surveillance, intelligence, computers to process the intelligence, and networks to transmit the intelligence between advisors in remote locations and their commanding officers in the Pentagon, Whitehouse, or elsewhere. Surveillance of this kind cannot be separated from either the creation of the ARPANET nor the emergence of the internet.

Conclusion

In examining the military origins of the internet we are confronted with a paradigm shift. Over a short span of time, we see how counterinsurgency and surveillance capabilities came to play a much greater role on the battlefield than trained soldiers. ARPA played a critical role in this shift. Indeed, its work on the ARPANET provided the ability to manage counterinsurgency and surveillance operations on an unprecedented scale. The networking technology it provided transformed the way in which enemies were monitored and studied, and the ways in which intelligence was gathered, processed and deployed. While ARPA outwardly preferred to talk up the academic benefits of the ARPANET, we cannot escape the military imperatives at the heart of its development.

Taking this a step further, the development of the ARPANET demonstrates how the boundaries between military and civilian life - already eroded by previous 20th century conflicts - took on a disturbing permanence during the Cold War era. Indeed, technology ostensibly developed for academic use proved highly effective in military operations. In turn, military operations provided the conditions needed for the further experimentation and refinement of the technology. In this context, the fine points of distinction between enemies on the battlefield and civilians back home seemed unimportant. The latter increasingly came to be seen as legitimate targets for counterinsurgency and mass surveillance.

To end on an optimistic note, we should also reflect on the importance of whistleblowers, testimony and exposé. At different points during our story, figures like Ford Rowan and Christopher Pyle were willing – some might even say brave – to call out what they saw as the immoral use of new technology against civil liberties. Their efforts to warn Americans of the dangers of mass surveillance and intelligence gathering played a part in holding those with power to account. This feels particularly important today. In recent years, we have seen the reach of the state dwarfed by the emergence of new players in the form of Big Tech. These companies, and the products and services they provide, have opened up new concerns about the way in which the internet works and who benefits from it. While these concerns are not likely to be resolved any time soon, it is worth remembering how people in the past sought to address the earliest ethical questions of the internet.

Find that piece of interest? If so, join us for free by clicking here.

References

Yasha Levine, Surveillance Valley: The Secret Military History of the Internet (2019)

In the Beginning, There Was Arpanet | Air & Space Forces Magazine (airandspaceforces.com)

Inside DARPA, The Pentagon Agency Whose Technology Has 'Changed the World' : NPR

The Emerging Digital Economy | U.S. Department of Commerce

https://en.wikipedia.org/wiki/Strategic_Hamlet_Program

How the internet was invented | Internet | The Guardian

INTERNET Prehistory: ARPANET Chronology

The Origin and Nature of the US “Military-Industrial Complex”