NSA all about Big Data

[rhoenix]

I'm putting this here, rather than in Science & Logic, due to it's effects.

Verizon has given the U.S. National Security Agency information on all its telephone calls for months. But it's not the calls' content the government is looking at—it's their context.

For the past three months, Verizon has handed over information on all telephone calls within its system to the U.S. government. This news, which broke last night thanks to a leaked court document, is a big story, but it isn't exactly a surprising one.

The Electronic Frontier Foundation, a group dedicated to the protection of fundamental rights online, has long suspected this kind of broad surveillance. Last summer, Senator Ron Wyden (D-Oregon), hinted that the government has broader surveillance powers than people suspect. The agencies doing the surveillance all fall under the executive branch, but congress has the power of oversight. Last December, Congress voted to extend the act granting this broad surveillance power until at least 2017.

What that all means: While the actual Verizon surveillance story is news, it's hardly unanticipated, and it falls into a much larger pattern of increased governmental surveillance powers post-9/11.

It also means that all of this is legal. The FBI had a warrant for the records it requested from Verizon, and, rather than break the law, Verizon obliged. Not everything was turned over to the government: Phone calls themselves are well protected by legal precedent, and obtaining warrants to listen to a tremendous amount of calls is much harder. Instead, the government asked for phone call metadata, which is kept and maintained by telecommunications companies. The metadata includes the time the call took place, the call origin, the call duration, and the carrier. In cellphones, it also includes the cell towers that transmitted the call, giving a rough approximation of the callers' physical location.

This information is all vulnerable because of a legal precedent set back in 1979. Phone call records were deemed not part of the protected private information of an individual, but instead the property of the telephone company. This problem resurfaced last month, when the Associated Press reported that the Department of Justice subpoenaed the call records for many of their writers as part of an ongoing leak investigation. It was a perfectly legal way for the government to obtain that information, but gathering metadata from calls is far more revealing now than it was in the 1970s.

Niraji Chokshi and Matt Berman note:

After analyzing 1.5 million cellphone users over the course of 15 months, the researchers found they could uniquely identify 95 percent of cellphone users based on just four data points—that is, just four instances of where they were and what hour of the day it was just four times in one year. With just two data points, they could identify more than half of the users.

It's conceivable that the National Security Agency is using this information from Verizon to track and identify every individual from their phone records. But that's a time- and data-intensive project. The request from Verizon dates to April 19, the date the surviving Boston bomber was arrested in Watertown, Mass. If there is a specific terrorist the NSA is trying to find, plotting the location of every single person by their phone records is the functional equivalent of combining a dozen haystacks into one great big pile with the hope of finding the one needle.

More likely is that this is a collection boon for general big data projects. The NSA has been mining phone records on a large scale since at least 2006, and a data set in the millions provides a tremendous resource for finding and discerning new patterns.

What new patterns? The NSA, by and large, has a mandate to collect intelligence on foreign communications and "foreign signals intelligence," which for our purposes involves electronic communication. There's a tremendous amount of information in the communications they regularly intercept, but because their focus is foreign, there's no clear control group for domestic communications that's different. The tremendous amount of data contained in the Verizon records gives the NSA a largely-domestic data set to work with, and then new information to refine their intelligence collection process. It gives them a baseline.

The Verizon data collection is fascinating tool set for spooks. Though perfectly legal, it's based on a 1970s understanding of phone record privacy. And the way it's going to be used is all 21st century.

And from another source:

The National Security Agency has obtained direct access to the systems of Google, Facebook, Apple and other US internet giants, according to a top secret document obtained by the Guardian.

The NSA access is part of a previously undisclosed program called PRISM, which allows them to collect material including search history, the content of emails, file transfers and live chats, the document says.

The Guardian has verified the authenticity of the document, a 41-slide Powerpoint presentation – classified as Top Secret with no distribution to foreign allies – which was apparently used to train intelligence operatives on the capabilities of the program. The document claims “collection directly from the servers” of major US service providers.

Although the presentation claims the program is run with the assistance of the companies, all those who responded to a Guardian request for comment on Thursday denied any knowledge of any such program.

In a statement, Google said: “Google cares deeply about the security of our users’ data. We disclose user data to government in accordance with the law, and we review all such requests carefully. From time to time, people allege that we have created a government ‘back door’ into our systems, but Google does not have a ‘back door’ for the government to access private user data.”

The NSA access, enabled by changes to US surveillance law introduced under President Bush and renewed under Obama in December 2012 .

The program facilitates extensive, in-depth surveillance on both live communications and stored information. The law allows for the targeting of any customers of participating firms who live outside the US, or those Americans whose communications include people outside the US.

It also opens the possibility of communications made entirely within the US being collected without warrants.

Disclosure of the PRISM program follows a leak to the Guardian on Wednesday of a top secret court order compelling telecoms provider Verizon to turn over the telephone records of millions of US customers.

The participation of the internet companies in PRISM will add to the debate, ignited by the Verizon revelation, about the scale of surveillance by the intelligence services. Unlike the collection of those call records, this surveillance can include the content of communications and not just the metadata.

Some of the world’s largest internet brands are claimed to be part of the information-sharing program since its introduction in 2007. Microsoft – which is currently running an advertising campaign with the slogan “Your privacy is our priority” – was the first, with collection beginning in December 2007.

It was followed by Yahoo in 2008; Google, Facebook and PalTalk in 2009; YouTube in 2010; Skype and AOL in 2011; and finally Apple, which joined the program in 2012. The program is continuing to expand, with other providers due to come online.

Collectively, the companies cover the vast majority of online email, search, video and communications networks.



The extent and nature of the data collected from each company varies.

Companies are legally obliged to comply with requests for users’ communications under US law, but the PRISM program allows the intelligence services direct access to the companies’ servers. The NSA document notes the operations have “assistance of communications providers in the US”.

The revelation also supports concerns raised by several US senators during the renewal of the Fisa Amendments Act in December 2012, who warned about the scale of surveillance the law might enable, and shortcomings in the safeguards it introduces.

When the FAA was first enacted, defenders of the statute argued that a significant check on abuse would be the NSA’s inability to obtain electronic communications without the consent of the telecom and internet companies that control the data. But the PRISM program renders that consent unnecessary, as it allows the agency to directly and unilaterally seize the communications off the companies’ servers

A chart prepared by the NSA, contained within the top secret document obtained by the Guardian, underscores the breadth of the data it is able to obtain: email, video and voice chat, videos, photos, voice-over-IP (for example, Skype) chats, file transfers, social networking details, and more.

The document is recent, dating to April 2013. Such a leak is extremely rare in the history of the NSA, which prides itself on maintaining a high level of secrecy.

The PRISM program allows the NSA, the world’s largest surveillance organisation, to obtain targeted communications without having to request them from the service providers and without having to obtain individual court orders.

With this program, the NSA is able to reach directly into the servers of the participating companies and obtain both stored communications as well as perform real-time collection on targeted users.

The presentation claims PRISM was introduced to overcome what the NSA regarded as shortcomings of Fisa warrants in tracking suspected foreign terrorists. It noted that the US has a “home field advantage” due to housing much of the internet’s architecture. But the presentation claimed “Fisa constraints restricted our ‘home field advantage’” because Fisa required individual warrants and confirmations that both the sender and receiver of a communication were outside the US.

“Fisa was broken because it provided privacy protections to people who were not entitled to them,” the presentation claimed. “It took a Fisa Court order to collect on foreigners overseas who were communicating with other foreigners overseas simply because the Government was collecting off a wire in the United States. There were too many e-mail accounts to be practical to seek Fisas for all.”

The new measures introduced in the FAA redefines “electronic surveillance” to exclude anyone “reasonably believed” to be outside the USA – a technical change which reduces the bar to initiating surveillace.

The act also gives the Director of National Intelligence and Attorney General power to permit obtaining intelligence information, and indemnifies internet companies against any actions arising as a result of co-operating with authorities’ requests.

In short, where previously the NSA needed individual authorisations, and confirmation that all parties were outside the USA, they now need only reasonable suspicion that one of the parties was outside the country at the time of the records were collected by the NSA.

The document also shows the FBI acts as an intermediary between other agencies and the tech companies, and stresses its reliance on the participation of US internet firms, claiming “access is 100% dependent on ISP provisioning”.

In the document, the NSA hails the PRISM program as “one of the most valuable, unique and productive accesses for NSA”.

It boasts of what it calls “strong growth” in its use of the PRISM program to obtain communications. The document highlights the number of obtained communications increased in 2012 by 248% for Skype – leading the notes to remark there was “exponential growth in Skype reporting; looks like the word is getting out about our capability against Skype”. There was also a 131% increase in requests for Facebook data, and 63% for Google.

The NSA document indicates that it is planning to add Dropbox as a PRISM provider. The agency also seeks, in its words, to “expand collection services from existing providers”.

The revelations echo fears raised on the Senate floor last year during the expedited debate on the renewal of the FAA powers which underpin the PRISM program, which occurred just days before the Act expired.

Senator Christopher Coons of Delaware specifically warned that the secrecy surrounding the various surveillance programs meant there was no way to know if safeguards within the act were working.

“The problem is we here in the United States Senate and so the citizens we represent don’t know how well any of these safeguards actually work,” he said.

“The law doesn’t forbid purely domestic information from being collected. We know that at least one Fisa court has ruled that the surveillance program violated the law. Why? Those who know can’t say and average Americans can’t know.”

Other senators also raised concerns. Senator Ron Wyden of Oregon attempted, without success, to find out any information on how many phone calls or emails had been intercepted under the program.

When the law was enacted, defenders of the FAA argued that a significant check on abuse would be the NSA’s inability to obtain electronic communications without the consent of the telecom and internet companies that control the data. But the PRISM program renders that consent unnecessary, as it allows the agency to directly and unilaterally seize the communications off the companies’ servers.

When the NSA reviews a communication it believes merits further investigation, it issues what it calls a “report”. According to the NSA, “over 2,000 PRISM-based reports” are now issued every month. There were 24,005 in 2012, a 27% increase on the previous year.

In total, more than 77,000 intelligence reports have cited the PRISM program.

Jameel Jaffer, director of the ACLU’s Center for Democracy, that it was astonishing the NSA would even ask technology companies to grant direct access to user data.

“It’s shocking enough just that the NSA is asking companies to do this,” he said. “The NSA is part of the military. The military has been granted unprecedented access to civilian communications.

“This is unprecedented militarisation of domestic communications infrastructure. That’s profoundly troubling to anyone who is concerned about that separation.”

We all knew this was happening in the background, but the scope of it is a little disquieting.

[rhoenix]

Additional source:

The scale of America's surveillance state was laid bare on Thursday as senior politicians revealed that the US counter-terrorism effort had swept up swaths of personal data from the phone calls of millions of citizens for years.

After the revelation by the Guardian of a sweeping secret court order that authorised the FBI to seize all call records from a subsidiary of Verizon, the Obama administration sought to defuse mounting anger over what critics described as the broadest surveillance ruling ever issued.

A White House spokesman said that laws governing such orders "are something that have been in place for a number of years now" and were vital for protecting national security. Dianne Feinstein, the Democratic chairwoman of the Senate intelligence committee, said the Verizon court order had been in place for seven years. "People want the homeland kept safe," Feinstein said.

But as the implications of the blanket approval for obtaining phone data reverberated around Washington and beyond, anger grew among other politicians.

Intelligence committee member Mark Udall, who has previously warned in broad terms about the scale of government snooping, said: "This sort of widescale surveillance should concern all of us and is the kind of government overreach I've said Americans would find shocking." Former vice-president Al Gore described the "secret blanket surveillance" as "obscenely outrageous".

The Verizon order was made under the provisions of the Foreign Intelligence Surveillance Act (Fisa) as amended by the Patriot Act of 2001, passed in the wake of the 9/11 attacks. But one of the authors of the Patriot Act, Republican congressman Jim Sensenbrenner, said he was troubled by the Guardian revelations. He said that he had written to the attorney general, Eric Holder, questioning whether "US constitutional rights were secure".

He said: "I do not believe the broadly drafted Fisa order is consistent with the requirements of the Patriot Act. Seizing phone records of millions of innocent people is excessive and un-American."

The White House sought to defend what it called "a critical tool in protecting the nation from terrorist threats". White House spokesman Josh Earnest said Fisa orders were used to "support important and highly sensitive intelligence collection operations" on which members of Congress were fully briefed.

"The intelligence community is conducting court-authorized intelligence activities pursuant to a public statute with the knowledge and oversight of Congress and the intelligence community in both houses of Congress," Earnest said.

He pointed out that the order only relates to the so-called metadata surrounding phone calls rather than the content of the calls themselves. "The order reprinted overnight does not allow the government to listen in on anyone's telephone calls," Earnest said.

"The information acquired does not include the content of any communications or the name of any subscriber. It relates exclusively to call details, such as a telephone number or the length of a telephone call."

But such metadata can provide authorities with vast knowledge about a caller's identity. Particularly when cross-checked against other public records, the metadata can reveal someone's name, address, driver's licence, credit history, social security number and more. Government analysts would be able to work out whether the relationship between two people was ongoing, occasional or a one-off.

The disclosure has reignited longstanding debates in the US over the proper extent of the government's domestic spying powers.

Ron Wyden of Oregon, a member of the Senate intelligence committee who, along with Udell, has expressed concern about the extent of US government surveillance, warned of "sweeping, dragnet surveillance". He said: "I am barred by Senate rules from commenting on some of the details at this time, However, I believe that when law-abiding Americans call their friends, who they call, when they call, and where they call from is private information.

"Collecting this data about every single phone call that every American makes every day would be a massive invasion of Americans' privacy."

Beyond Orwellian

Jameel Jaffer, deputy legal director at the American Civil Liberties Union, said: "From a civil liberties perspective, the program could hardly be any more alarming. It's a program in which some untold number of innocent people have been put under the constant surveillance of government agents.

"It is beyond Orwellian, and it provides further evidence of the extent to which basic democratic rights are being surrendered in secret to the demands of unaccountable intelligence agencies."

Under the Bush administration, officials in security agencies had disclosed to reporters the large-scale collection of call records data by the NSA, but this is the first time significant and top-secret documents have revealed the continuation of the practice under President Obama.

The order names Verizon Business Services, a division of Verizon Communications. In its first-quarter earnings report, published in April, Verizon Communications listed about 10 million commercial lines out of a total of 121 million customers. The court order, which lasts for three months from 25 April, does not specify what type of lines are being tracked. It is not clear whether any additional orders exist to cover Verizon's wireless and residential customers, or those of other phone carriers.

Fisa court orders typically direct the production of records pertaining to a specific, named target suspected of being an agent of a terrorist group or foreign state, or a finite set of individually named targets. The unlimited nature of the records being handed over to the NSA is extremely unusual.

Feinstein said she believed the order had been in place for some time. She said: "As far as I know this is the exact three-month renewal of what has been the case for the past seven years. This renewal is carried out by the [foreign intelligence surveillance] court under the business records section of the Patriot Act. Therefore it is lawful. It has been briefed to Congress."

The Center for Constitutional Rights said in a statement that the secret court order was unprecedented. "As far as we know this order from the Fisa court is the broadest surveillance order to ever have been issued: it requires no level of suspicion and applies to all Verizon [business services] subscribers anywhere in the US.

"The Patriot Act's incredibly broad surveillance provision purportedly authorizes an order of this sort, though its constitutionality is in question and several senators have complained about it."

Russell Tice, a retired National Security Agency intelligence analyst and whistleblower, said: "What is going on is much larger and more systemic than anything anyone has ever suspected or imagined."

Although an anonymous senior Obama administration official said that "on its face" the court order revealed by the Guardian did not authorise the government to listen in on people's phone calls, Tice now believes the NSA has constructed such a capability.

"I figured it would probably be about 2015" before the NSA had "the computer capacity … to collect all digital communications word for word," Tice said. "But I think I'm wrong. I think they have it right now."

[rhoenix]

Yet another article:

Yesterday's revelations from The Guardian about US government collection of data on just about every phone call made in the country is exactly the kind of scenario civil libertarians have been fearing: widespread, dragnet-style surveillance of every American's telephone calls.

To put it simply: the published Verizon document suggests the US government has been keeping a log of just about every phone call made in the last seven years, full stop. This includes who the call was from, who it was made to, how long it was, and where the caller was when the conversation happened.

The responses today have been almost as stunning as the revelation itself. The White House called the data collection a "critical" tool in the fight against terror and noted that the "information acquired does not include the content of any communications or the name of any subscriber." Of course, that's a thin obfuscation; the dragnet is merely a first step, because that's how dragnets work. Once the government determines someone is calling a person of interest, it could easily begin the process of wiretapping that person and collecting personal information, whether the target is a terrorist, or a journalist, activist, or lawyer.

And while activists strongly suspected what was going on, the US Senate knew what was going on—and saw surprisingly little dissent. Two who did protest were Sens. Ron Wyden (D-OR) and Mark Udall (D-CO).

"When the American people find out how their government has secretly interpreted the Patriot Act, they will be stunned and they will be angry," Wyden warned in 2011. Udall said Americans would be "alarmed" if they knew how the Patriot Act was being interpreted to allow mass spying.

Today Udall told his hometown paper, The Denver Post, that he indeed knew about the NSA wiretapping. Udall said he “did everything short of leaking classified information” to stop it and wants to hold Obama accountable for his promises to increase transparency.

The facts suggested Udall and Wyden were lone voices in the wilderness in 2011, and now that the details of the program are public and being responded to, they look even more alone. Senators from both parties have jumped to defend the mass data-collection.

"Everyone should just calm down," Senate Majority Leader Harry Reid (D-NV) told reporters today. He emphasized that the published order is just a renewal of routine data collection that's been going on for seven years (as if that makes the situation better).

Sen. Diane Feinstein (D-CA), who chairs the Senate Intelligence Committee, also vocally defended the program today. In her view, the program was duly authorized by the "business records" provision of the Patriot Act. "Therefore, it is lawful," said Feinstein. "It has been briefed to Congress.”

The ranking Republican who presides over Senate Intelligence with Feinstein is Saxby Chambliss (D-GA). Chambliss today spoke to The New York Times about how the program is used:

It’s metadata only and it’s what we call minimized. All of these numbers are basically ferreted out by a computer, but if there’s a number that matches a terrorist number that has been dialed by a US number or dialed from a terrorist to a US number, then that may be flagged. And they may or may not seek a court order to go further on that particular instance. But that’s the only time that this information is ever used in any kind of substantive way.”

Of course, it's a matter of interpretation whether or not the Patriot Act really allows the kind of mass data collection we now know is being undertaken. So how does the government interpret the Patriot Act?

It won't say. Udall and Wyden tried to force the Obama Administration to describe how it was interpreting the Patriot Act in 2011, but they couldn't push through their amendment. That same year, the Electronic Frontier Foundation (EFF) filed a Freedom of Information Act (FOIA) lawsuit demanding to get documents about the government's secret interpretation of Section 215 of that act, the section that allows it to collect records like the "telephony metadata" Verizon has been handing over.

At a March hearing in that case, the government wouldn't even provide page numbers of the documents relevant to EFF's request. Under judicial pressure, the government has now provided the number of pages on some documents. On April 18, DOJ lawyers made a more detailed filing that includes dates and page lengths for most, but not all, of the 101 documents EFF is seeking access to. Some of documents are identified with ridiculous date ranges, like "2/2006-2/2011."

There's also tantalizing descriptions of documents. "Training materials for government personnel pertaining to implementation of section 215 authority," reads the description of several documents.

That case is now lined up for summary judgment. If the EFF wins, the government will surely appeal (and appeal again if necessary); a recent 5-4 Supreme Court surveillance-related decision doesn't bode well for court challenges to government secrecy.

While it's hard to imagine this case bringing much transparency, public knowledge of the government actions and justifications described in those documents is more important now than ever.

But in the past decade, the courts have proved an exceptionally weak tool for those challenging the constitutionality of this brave new world of surveillance. Another EFF case, which was premised on the (correct) assumption that dragnet surveillance was taking place, has been stalled by government arguments about state secrecy. Indeed, one key leak and the political discussion that has ensued has shed more light on what's going on than years of litigation.