Since 2010, the National Security Agency has been exploiting its huge collections of data to create sophisticated graphs of some Americans’ social connections that can identify their associates, their locations at certain times, their traveling companions and other personal information, according to newly disclosed documents and interviews with officials. http://www.nytimes.com/2013/09/29/us/nsa-examines-social-networks-of-us-citizens.html?pagewanted=1&_r=2
The spy agency began allowing the analysis of phone call and e-mail logs in November 2010 to examine Americans’ networks of associations for foreign intelligence purposes after N.S.A. officials lifted restrictions on the practice, according to documents provided by Edward J. Snowden, the former N.S.A. contractor.
The agency was authorized to conduct “large-scale graph analysis on very large sets of communications metadata without having to check foreignness” of every e-mail address, phone number or other identifier, the document said.
The agency can augment the communications data with material from public, commercial and other sources, including bank codes, insurance information, Facebook profiles, passenger manifests, voter registration rolls and GPS location information, as well as property records and unspecified tax data, according to the documents. They do not indicate any restrictions on the use of such “enrichment” data, and several former senior Obama administration officials said the agency drew on it for both Americans and foreigners.
N.S.A. officials declined to say how many Americans have been caught up in the effort, including people involved in no wrongdoing. The documents do not describe what has resulted from the scrutiny, which links phone numbers and e-mails in a “contact chain” tied directly or indirectly to a person or organization overseas that is of foreign intelligence interest.
The new disclosures add to the growing body of knowledge in recent months about the N.S.A.’s access to and use of private information concerning Americans, prompting lawmakers in Washington to call for reining in the agency and President Obama to order an examination of its surveillance policies. Almost everything about the agency’s operations is hidden, and the decision to revise the limits concerning Americans was made in secret, without review by the nation’s intelligence court or any public debate. As far back as 2006, a Justice Department memo warned of the potential for the “misuse” of such information without adequate safeguards.
N.S.A. officials declined to identify which phone and e-mail databases are used to create the social network diagrams, and the documents provided by Mr. Snowden do not specify them. The agency did say that the large database of Americans’ domestic phone call records, which was revealed by Mr. Snowden in June and caused bipartisan alarm in Washington, was excluded.
But the agency has multiple collection programs and databases, the former officials said, adding that the social networking analyses relied on both domestic and international metadata.
A series of agency PowerPoint presentations and memos describe how the N.S.A. has been able to develop software and other tools — one document cited a new generation of programs that “revolutionize” data collection and analysis — to unlock as many secrets about individuals as possible.
N.S.A. analysts can exploit that information to develop a portrait of an individual, one that is perhaps more complete and predictive of behavior than could be obtained by listening to phone conversations or reading e-mails, experts say.
Phone and e-mail logs, for example, allow analysts to identify people’s friends and associates, detect where they were at a certain time, acquire clues to religious or political affiliations, and pick up sensitive information like regular calls to a psychiatrist’s office, late-night messages to an extramarital partner.
“Metadata can be very revealing,” said Orin S. Kerr, a law professor at George Washington University. “Knowing things like the number someone just dialed or the location of the person’s cellphone is going to allow them to assemble a picture of what someone is up to. It’s the digital equivalent of tailing a suspect.”
The agency had asked for new power as early as 1999, the documents show, but had been initially rebuffed because it was not permitted under rules of the Foreign Intelligence Surveillance Court that were intended to protect the privacy of Americans.
A 2009 draft of an N.S.A. inspector general’s report suggests that contact chaining and analysis may have been done on Americans’ communications data under the Bush administration’s program of wiretapping without warrants, which began after the Sept. 11 attacks to skirt the existing laws governing electronic surveillance.
The N.S.A. performed the social network graphing in a pilot project for 1 ½ years “to great benefit,” according to the 2011 memo. It was put in place in November 2010 in “Sigint Management Directive 424” (sigint refers to signals intelligence).
The N.S.A. documents show that one of the main tools used for chaining phone numbers and e-mail addresses has the code name Mainway. It is a repository into which vast amounts of data flow daily from the agency’s fiber-optic cables, corporate partners and foreign computer networks that have been hacked.
The documents show that significant amounts of information from the United States go into Mainway. An internal N.S.A. bulletin, for example, noted that in 2011 Mainway was taking in 700 million phone records per day. In August 2011, it began receiving an additional 1.1 billion cellphone records daily from an unnamed American service provider under Section 702 of the 2008 FISA Amendments Act, which allows for the collection of the data of Americans if at least one end of the communication is believed to be foreign.
The overall volume of metadata collected by the N.S.A. is reflected in the agency’s secret 2013 budget request to Congress. The budget document, disclosed by Mr. Snowden, shows that the agency is pouring money and manpower into creating a metadata repository capable of taking in 20 billion “record events” daily and making them available to N.S.A. analysts within 60 minutes.
The spending includes support for the “Enterprise Knowledge System,” which has a $394 million multiyear budget and is designed to “rapidly discover and correlate complex relationships and patterns across diverse data sources on a massive scale,” according to a 2008 document. The data is automatically computed to speed queries and discover new targets for surveillance.
A top-secret document titled “Better Person Centric Analysis” describes how the agency looks for 94 “entity types,” including phone numbers, e-mail addresses and IP addresses. In addition, the N.S.A. correlates 164 “relationship types” to build social networks and what the agency calls “community of interest” profiles, using queries like “travelsWith, hasFather, sentForumMessage, employs.”
A 2009 PowerPoint presentation provided more examples of data sources available in the “enrichment” process, including location-based services like GPS and TomTom, online social networks, billing records and bank codes for transactions in the United States and overseas.
At a Senate Intelligence Committee hearing on Thursday, General Alexander was asked if the agency ever collected or planned to collect bulk records about Americans’ locations based on cellphone tower data. He replied that it was not doing so as part of the call log program authorized by the Patriot Act, but said a fuller response would be classified.
If the N.S.A. does not immediately use the phone and e-mail logging data of an American, it can be stored for later use, at least under certain circumstances, according to several documents.
One 2011 memo, for example, said that after a court ruling narrowed the scope of the agency’s collection, the data in question was “being buffered for possible ingest” later. A year earlier, an internal briefing paper from the N.S.A. Office of Legal Counsel showed that the agency was allowed to collect and retain raw traffic, which includes both metadata and content, about “U.S. persons” for up to five years online and for an additional 10 years offline for “historical searches.”