Democratic Underground Latest Greatest Lobby Journals Search Options Help Login
Google

NSA analyst ‘improperly accessed’ Bill Clinton’s e-mail through domestic surveillance program.

Printer-friendly format Printer-friendly format
Printer-friendly format Email this thread to a friend
Printer-friendly format Bookmark this thread
This topic is archived.
Home » Discuss » Archives » General Discussion (1/22-2007 thru 12/14/2010) Donate to DU
 
babylonsister Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 10:05 AM
Original message
NSA analyst ‘improperly accessed’ Bill Clinton’s e-mail through domestic surveillance program.
http://thinkprogress.org/2009/06/17/nsa-bill-clinton-email/

NSA analyst ‘improperly accessed’ Bill Clinton’s e-mail through domestic surveillance program.

The New York Times reports today that members of Congress are increasingly concerned about the extent of the NSA’s domestic surveillance program, particularly the overcollection of the private telephone calls and e-mail messages of Americans. An anonymous former intelligence analyst tells reporters James Risen and Eric Lichtblau that during much of the Bush years, the NSA “tolerated significant collection and examination of domestic e-mail messages without warrants.” Reportedly, one of the accessed domestic e-mail accounts belonged to former President Bill Clinton:

He said he and other analysts were trained to use a secret database, code-named Pinwale, in 2005 that archived foreign and domestic e-mail messages. He said Pinwale allowed N.S.A. analysts to read large volumes of e-mail messages to and from Americans as long as they fell within certain limits — no more than 30 percent of any database search, he recalled being told — and Americans were not explicitly singled out in the searches.

The former analyst added that his instructors had warned against committing any abuses, telling his class that another analyst had been investigated because he had improperly accessed the personal e-mail of former President Bill Clinton.


Article here:

http://www.democraticunderground.com/discuss/duboard.php?az=show_mesg&forum=389&topic_id=5866668&mesg_id=5866668
Printer Friendly | Permalink |  | Top
RandomThoughts Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 10:08 AM
Response to Original message
1. So why are they not talking about the email of millions of other Americans.
Does the private citizen Bill Clinton have more rights then other citizens?

What about journalist, or activist, or people looking at a career in public office.

The whole program without oversight is pathetic.
Printer Friendly | Permalink |  | Top
 
Eric J in MN Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 10:12 AM
Response to Reply #1
3. If the article didn't name anyone famous, then some people would figure...
...that the Americans being spied on were terrorists.
Printer Friendly | Permalink |  | Top
 
Winterblues Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 11:36 AM
Response to Reply #3
9. Don't the wing nuts consider Clinton and in fact all Liberals to be terrorists?
:shrug:
Printer Friendly | Permalink |  | Top
 
Echo In Light Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 11:43 AM
Response to Reply #1
10. Come now...EVERYONE knows that's just a "conspiracy theory!"
Printer Friendly | Permalink |  | Top
 
Eric J in MN Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 10:09 AM
Response to Original message
2. If only there were lawsuits to find out the extent of government spying.
Edited on Wed Jun-17-09 10:10 AM by Eric J in MN
Wait, there were, but Congress gave retroactive immunity to phone companies and blocked those lawsuits with the FISA bill.
Printer Friendly | Permalink |  | Top
 
Captain Hilts Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 10:15 AM
Response to Original message
4. In 1943 the Army made the mistake of bugging Eleanor R's hotel room. MISTAKE. nt
Printer Friendly | Permalink |  | Top
 
Sanity Claws Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 07:56 PM
Response to Reply #4
15. I don't know about this incident
What happened?
Printer Friendly | Permalink |  | Top
 
gratuitous Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 10:19 AM
Response to Original message
5. Oh, it's just an "overcollection" of calls and messages
Jesus, they must have a full-time staff thinking up these phrases and words. Anything to avoid words like "crime" and "spying" and "illegal as all hell."
Printer Friendly | Permalink |  | Top
 
peacetalksforall Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 09:50 PM
Response to Reply #5
23.  ... they must have a full-time staff thinking up these phrases and words. Peggy Noonan.
Printer Friendly | Permalink |  | Top
 
bertman Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 10:29 AM
Response to Original message
6. They were probably trying to get phone numbers for some of Bubba's side squeezes.
That's legal isn't it?

:evilgrin:
Printer Friendly | Permalink |  | Top
 
Ohio Joe Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 11:19 AM
Response to Original message
7. A few thoughts on this from a programmers point of view
OK, let me try and take this from the start and see if I can explain how I expect this is working. There has to be some type of scanning program that checks email while "in transit", keeping a copy if it meets whatever criteria they use. There is no way they are keeping every email sent, it just would not work.

1 - space issues. The entire contents of the email has to be in the database for any scans to do anything. Since emails are not in any standard data format, scanning programs would have to parse through each email, one byte at a time, looking for their key words or phrases.

2 - Time. Yes, computers are fast but if they had every email from everyone, queries would take weeks just to return a crapload of results that then have to be reviewed by people. Now, if I were to design such an abomination, I'd put in a switch that said the email had already been scanned/reviewed but I/O would still happen to check it and it is the I/O that causes these things to take a lot of time to run.

3 - Manpower. I don't care what key words they use, a shit load of results are going to be had and need a pretty bug army to review.

So... as I said, they are probably scanned while in transit by a fairly simple program, can't be too complex or all encompassing or they bring email to a halt and pretty much everyone would be seeing it. Once an email is selected to be reviewed, it would then be stored in the database where it could be scanned more closely by a more complex program. Anything that can be eliminated in this second go through is almost certainly deleted (without back-up) for the same reasons as above. Then you would have a group of people (probably still an army) that would have to manually go through whats left to see if there are any real threats there.

You ready for the kicker?

Its pointless. They will not find a single actionable thing. No way.

Think about it. Back in the late 70's, early 80's, when I smoked a lot of dope, everyone knew you don't talk about it on the phone... just in case. So, I'd call my guy and say "Hey man! I was thinking of stopping over, you going to be around?" He was my fucking dealer, he knew what I wanted, we did not have set code words we used all the time. We knew what we were talking about and made shit up on the fly.

Terrorist-1 is not going to send an email to terrorist-2 and say "Hey! Don't forget the bomb for tonights attack, you know you always forget the bomb! We will meet at main street in NY and proceed to town hall and blow it up (if you remember the bomb hahaha)". No... just no. They know what they are doing and it will more likely say "Hey! see you at the bar tonight!" Details are not going to be sent through email, specific discussion of the plot is not going to be discussed in email.

This is, simply put, a stupid plan. It is a waste of money, time, manpower and a clear violation of people privacy. The only way looking at someones email is going to yield results is if you already know they are involved in a plot. Then when you read the email, you also KNOW what they are talking about.
Printer Friendly | Permalink |  | Top
 
libertypirate Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 11:32 AM
Response to Reply #7
8. You shouldn't be postulating when their solution is already public
NarusInsight is the most scalable traffic intelligence system for capturing, analyzing and correlating IP traffic in real time.

NarusInsight is the foundation for the industry's most advanced real-time cyber protection, intercept and traffic management solutions. Beginning with ubiquitous collection directly from the network links or from the elements themselves and moving through a massively scalable processing engine, NarusInsight has the ability to simultaneously provide deep-packet analysis and full traffic correlation across every link and every element on the network. Regardless of the consuming application and whether the information required is based on full packets or application-layer metrics or basic flow-level information, NarusInsight captures and processes the information in real time and as part of a single system.

Traditional legacy, siloed security, lawful intercept and network management solutions fail to scale. NarusInsight offers a massively scalable and distributed processing system that is easily extended by sophisticated Traffic Intelligence Modules, which provide the essential insight required to support both the rapid deployment of new services and smooth secure delivery of current services.

http://narus.com/index.php/product/insight-traffic
http://narus.com/

Printer Friendly | Permalink |  | Top
 
Ohio Joe Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 11:45 AM
Response to Reply #8
11. Why not? It appears I was correct
This package does a lite scan of email "in transit" and directs it to another place to be analyzed based on user defined parameters. I just tried to explain it in a manner most people could understand instead of giving a link to technobabble a lot of people may not understand.
Printer Friendly | Permalink |  | Top
 
defendandprotect Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 08:01 PM
Response to Reply #7
17. Hi Ohio . . . I know nothing about computers, but nonetheless offer this . . .
They aren't looking for terrorists because they are the terrorists - See: Bushco

There is probably an "enemies list" -- a very, very long one -- such as Nixon had --
and obviously name directed. Probably reviewed by higher ups. Possibly even including
friends and family and lawyers of enemy.

Since they have no idea what they will ever want or need, I imagine they are keeping
everything.

The idea isn't to read now. The idea is that you represent a problem later and they
pull your data.

I imagine that in tapping enemies one interesting thing leads to another until all of
DC is suddenly being read one morning. Yes . . . a large "bug" army is needed.

Plus there would be other valuable info -- financial, etc. -- if they can figure it out.
And the personal could also be valuable -- i.e., divorce, marriage, children, schools,
vacations, lovers - on and on!

Again, they're not looking for terrorists - they are the terrorists.





Printer Friendly | Permalink |  | Top
 
Ohio Joe Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 08:28 PM
Response to Reply #17
19. Well...
I would not put it past bushco to do such a thing. In terms of could it be done, absolutely, not a problem at all. The only thing I would have a problem with is that now that bushco is no longer calling the shots, they would no longer have access to the information. That would leave President Obama's administration with this mechanism to store enemies emails for 'dirty trick' purposes and honestly... I don't think he needs it. I mean, the repugs seem to be self destructing without any help.
Printer Friendly | Permalink |  | Top
 
defendandprotect Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 09:06 PM
Response to Reply #19
21. Are we sure they no longer have access to the info . . . ???
Reminds me of the time when Carter was president, but other people seemed

to be controlling it all, to Carter's dismay.

No . . . I think the game then reverses . . . checking to see who they were

interested in, what they found out. And, thinking about what you might now

like to find out. Even when your enemy may be "self-destructing" you still

usually want to know for sure and whatever else you can.'

It's power -- and Repugs aren't alone in wanting it. Everyone has a question

they want answered.

:)

Hope you're right and I'm wrong!

Printer Friendly | Permalink |  | Top
 
peacetalksforall Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 10:28 PM
Response to Reply #7
24. Thanks. However, I never considered the database to have a primary purpose of finding terrorists.
I believed it to be way to keep an eye on domestic political enemies, a way to gain data that could be used in a blackmail, and to populate what I call the Poindexter db. Your three points are excellent to demonstrated the futility of serious terrorist searching.

It's th eiceberg principle. They told us they were looking at what was above the water line. They really wanted a catch of data on what was below for their personal and party gain and control.

All screws go to AT & T, plus the others second whether we know their name of not.
Printer Friendly | Permalink |  | Top
 
Ohio Joe Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 11:57 PM
Response to Reply #24
25. I don't know
On a technical level, it most definitely could be done. I base my guesses here on the article in the OP. I readily admit, I find no point to the exercise as stated in the story but... having worked both for (the state of Ohio, in Child Services IT dept.) and with (as a corporate IT rep with Nationwide Insurance... ugh DMV crap) the government, I'm not one bit shocked they would be attempting exactly what is in the story. Welcome to IT, we do weird shit :D
Printer Friendly | Permalink |  | Top
 
malaise Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 07:44 PM
Response to Original message
12. Kick n/t
Printer Friendly | Permalink |  | Top
 
defendandprotect Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 07:45 PM
Response to Original message
13. Was this current Clinton mail . . . or stuff from his presidency...or all of it???
Printer Friendly | Permalink |  | Top
 
Duer 157099 Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 07:52 PM
Response to Original message
14. Let me remind everyone of what's really important here
1) If, as is almost certain, they have all of these emails saved and archived somewhere, ALL of us should be concerned just because of how probable it is that eventually they will end up in "wrong" hands. Could be hackers from China, whatever, it doesn't matter. What matters is that the possibility (certainty) exists.

2) Given 1) we should all be concerned about how these emails could be used for a) blackmail (by whoever against whoever) b) corporate espionage and c) datamining (by whoever for whatever reason)

Just because the technology may not exist now to effectively use the data is nothing to feel comfortable about. If you don't know that within a few years, that technology WILL be developed, then you're not paying attention.

Anyone who thinks to themselves "I have nothing to hide" just plain does not understand the issue at all.

There are huge implications.
Printer Friendly | Permalink |  | Top
 
Octafish Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 08:01 PM
Response to Reply #14
16. Frank Church knew and he warned us. Seems like the wrong people listened.
Frank Church and the Abyss of Warrantless Wiretapping.

Thanks, Duer 157099, for keeping alive the Spirit of '76.
Printer Friendly | Permalink |  | Top
 
defendandprotect Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 08:13 PM
Response to Reply #16
18. Evidently, the Church Report has never been released . . . !!!!
Keep in mind that Nixon was bugging everyone -- including Kissinger's bedroom!
In fact, I think he bugged Kissinger's pj's!

Eleanor Roosevelt was right -- and there is no denying it -- these are Gestapo
tactics.

Frank Church was targeted after this by CIA to prevent his relection -- as I recall
the story.

IMO, Church probably had a pretty firm idea of the CIA involvement in the JFK coup -
of course they didn't do it alone. According to Madelaine Brown it was being planned
for two years! Many right-wingers were involved. Many still are.

Thanks for the reminder -- we need to remember Frank Church and understand what he
tried so hard to do for us.



Printer Friendly | Permalink |  | Top
 
Born_A_Truman Donating Member (1000+ posts) Send PM | Profile | Ignore Thu Jun-18-09 12:20 AM
Response to Reply #14
26. Hackers?
What makes you think only the US has the technology?
Printer Friendly | Permalink |  | Top
 
leveymg Donating Member (1000+ posts) Send PM | Profile | Ignore Thu Jun-18-09 06:09 AM
Response to Reply #26
27. Bingo. Narus is an Israeli company.
So is Verint, another major supplier of CALEA compliance tapping and analysis systems.
Printer Friendly | Permalink |  | Top
 
tom_paine Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 08:33 PM
Response to Original message
20. Translation: Corrupt Bushie in NSA got caught.
Funny thing is: Bushies caught in corruption always lie in this fashion and always get away with it.

Sometimes the Democratic Leadership itself kills the invetigation, such as the 2006 invetsigation in FL-13 where the Bushies were caught red-handed and the whole edifice would have craced wide open, I believe, with some investigation.

The Democratic Leadership made sure the investigation never happened. Feel free to Google it.

One wonders, given the corrupted nature of American Imperial politics, if the investigation was killed BECAUSE it would have blown the Bushies' lection Fraud Ops wide open.

Who knows anymore? Nothing at this point would surprise me.
Printer Friendly | Permalink |  | Top
 
peacetalksforall Donating Member (1000+ posts) Send PM | Profile | Ignore Wed Jun-17-09 09:49 PM
Response to Original message
22. I would never buy an AT & T product nor accept a free pencil. Same for Verizon
and Southern Bell. How many others? I also think Microsoft aided and abetted the invasion of privacy. Only a feeling. No links.
Printer Friendly | Permalink |  | Top
 
leveymg Donating Member (1000+ posts) Send PM | Profile | Ignore Thu Jun-18-09 06:25 AM
Response to Original message
28. Pinwale was what Comey and Gonzalez tussled over - one of 10 secret NSA programs
Edited on Thu Jun-18-09 06:27 AM by leveymg
http://politics.theatlantic.com/2009/06/pinwale_and_the_new_nsa_revelations.php - Cached - Similar


Jun 16 2009, 9:39 pm by Marc Ambinder

Pinwale And The New NSA Revelations

The New York Times' Pulitzer-Prize-winning duo of Eric Lichtblau and James Risen have new details about the extent of National Security Agency surveillance of electronic communications in the United States. They reveal that a dispute over e-mail monitoring of Americans was at the heart of a now famous confrontation between then-acting Attorney General James Comey and White House counsel Alberto Gonzales. L and R refer to the program as having the code name of "Pinwale." Pinwale, though, is actually an unclassified proprietary term used to refer to advanced data-mining software that the government uses. Contractors who do SIGINT mining work often include a familiarity with Pinwale as a prerequesite for certain jobs. To keep things straight, the American public now has confirmation of at least four separate NSA domestic surveillance programs. The first is the Terrorist Surveillance Program, which involves the monitoring of telephone calls. The second is "Stellar Wind," a code name for a program that involves meta-data mining. The third is a program that keeps tabs on all the information that flows through telecom hubs under the control of U.S. companies and within the U.S. The fourth is the Pinwale e-mail exploitation. Obviously, these programs overlap. A marvelous new book about the NSA, The Secret Sentry, reports that there are at least ten separate new counterterrorist programs that are segregated from the rest of the NSA's highly classified programs and activities. Four down, six to go. (For more on the NSA's activities, please read my colleague Shane Harris's collected works.)linked



Printer Friendly | Permalink |  | Top
 
leveymg Donating Member (1000+ posts) Send PM | Profile | Ignore Thu Jun-18-09 06:54 AM
Response to Reply #28
29. STAR - related FBI terrorist profiling and data mining system
Edited on Thu Jun-18-09 06:56 AM by leveymg
http://www.washingtonpost.com/wp-dyn/content/article/2007/07/10/AR2007071001871.html
FBI Plans Initiative To Profile Terrorists
Potential Targets Get Risk Rating


By Ellen Nakashima
Washington Post Staff Writer
Wednesday, July 11, 2007; Page A08

The Federal Bureau of Investigations is developing a computer-profiling system that would enable investigators to target possible terror suspects, according to a Justice Department report submitted to Congress yesterday.

The System to Assess Risk, or STAR, assigns risk scores to possible suspects based on a variety of information, similar to the way a credit bureau assigns a rating based on a consumer's spending behavior and debt. The program focuses on foreign suspects but also includes data about some U.S. residents. A prototype is expected to be tested this year.

Justice Department officials said the system offers analysts a powerful new tool for finding possible terrorists. They said it is an effort to automate what analysts have been doing manually.

"STAR does not label anyone a terrorist," the report said. "Only individuals considered emergent foreign threats (as opposed to other criminal activity such as U.S. bank robbery threats) will be analyzed."

Some lawmakers said, however, that the report raises new questions about the government's power to use personal information and intelligence without accountability.


"The Bush administration has expanded the use of this technology, often in secret, to collect and sift through Americans' most sensitive personal information," said Sen. Patrick J. Leahy (D-Vt.), chairman of the Senate Judiciary Committee, which received a copy of the report on data-mining initiatives.

The use of data mining in the war on terror has sparked criticism. An airplane-passenger screening program called CAPPS II was revamped and renamed because of civil liberty concerns. An effort to collect Americans' personal and financial data called Total Information Awareness was killed.

Law enforcement and national security officials have continued working on other programs to use computers to sift through information for signs of threats. The Department of Homeland Security, for example, flags travelers entering and leaving the United States who may be potential suspects through a risk-assessment program called the Automated Targeting System.

STAR is being developed by the FBI's Foreign Terrorist Tracking Task Force, which tracks suspected terrorists inside the country or as they enter.

Both the Department of Homeland Security and the FBI's STAR programs create their ratings based on certain rules. In the case of STAR, a person's score would increase if his or her name matches one on a terrorist watch list, for example. A country of origin could also be weighted in a person's score.

After STAR has received the names of persons of interest, it runs them through an FBI "data mart" that includes classified and unclassified information from the government, airlines and commercial data brokers such as ChoicePoint. Then it runs them through the terrorist screening center database, which contains hundreds of thousands of names, as well as through a database containing information on non-citizens who enter the country. It also runs the names against information provided by data broker Accurint, which tracks addresses, phone numbers and driver's licenses.

The report said access to STAR would be limited to trained users and that data would be obtained lawfully. Results would be kept within the FBI's terrorist task force, the report said.

Privacy expert David Sobel, senior counsel for the nonprofit advocacy group Electronic Frontier Foundation, said the government's system depends on potentially unreliable data. "If we can't assess the accuracy of the information being fed into the system, it's very hard to assess the effectiveness of the system."

The STAR system would be subject to a privacy-impact assessment before launched in final form.





Printer Friendly | Permalink |  | Top
 
leveymg Donating Member (1000+ posts) Send PM | Profile | Ignore Thu Jun-18-09 07:12 AM
Response to Reply #28
30. 500 related programs and tools described and listed by Arkin:

Telephone Records are just the Tip of NSA's Iceberg
William M. Arkin on National and Homeland Security

http://blog.washingtonpost.com/early...the.html#20161

The National Security Agency and other U.S. government organizations
have developed hundreds of software programs and analytic tools to
"harvest" intelligence, and they've created dozens of gigantic
databases designed to discover potential terrorist activity both inside
the United States and overseas.
These cutting edge tools -- some highly classified because of their
functions and capabilities -- continually process hundreds of billions
of what are called "structured" data records, including telephone call
records and e-mail headers contained in information "feeds" that have
been established to flow into the intelligence agencies.
The multi-billion dollar program, which began before 9/11 but has been
accelerated since then. Well over 100 government contractors have
participated, including both small boutique companies whose products
include commercial off-the-shelf software and some of the largest
defense contractors, who have developed specialized software and tools
exclusively for government use.
USA Today provided a small window into this massive intelligence
community program by reporting yesterday that the NSA was collecting
and analyzing millions of telephone call records.
The call records are "structured data," that is, information maintained
in a standardized format that can be easily analyzed by machine
programs without human intervention. They're different from intercepts
of actual communication between people in that they don't contain the
"content" of the communications -- content that the Supreme Court has
ruled is protected under the Fourth Amendment. You can think of call
records as what's outside the envelope, as opposed to what's on the
inside.
Once collected, the call records and other non-content communication
are being churned through a mind boggling network of software and data
mining tools to extract intelligence. And this NSA dominated program of
ingestion, digestion, and distribution of potential intelligence raises
profound questions about the privacy and civil liberties of all
Americans.
Although there is no evidence that the harvesting programs have been
involved in illegal activity or have been abused to reach into the
lives of innocent Americans, their sheer scope, the number of
"transactions" being tracked, raises questions as to whether an
all-seeing domestic surveillance system isn't slowly being established,
one that in just a few years time will be able to reveal the
interactions of any targeted individual in near real time.
In late November 1998, the intelligence community and the Department of
Defense established the Advanced Research and Development Activity in
Information Technology (ARDA), a government consortium charged with
incubating and developing "revolutionary" research and development in
the field of intelligence processing.
The Director of the National Security Agency (NSA) agreed to establish,
as a component of the NSA, an organizational unit to carry out the
functions of ARDA, overseeing the research program of the CIA, DIA,
National Reconnaissance Office, and other defense and civilian
intelligence agencies.
Beginning before 9/11, ARDA established an "information exploitation"
program to fund and focus private research on operationally-relevant
problems of exploiting the increasing torrents of digital data
available to the intelligence community. Even with thousands of
analysts, NSA and other agencies were falling behind in their ability
to handle the volume of incoming material. Existing mainframe machine
aided processes were also falling behind advances in information
processing, particularly as the cost of computing power dramatically
declined in the 1990s.
The information exploitation research program has funded hundreds of
projects to find better ways to "pull" information, "push" information,
and "navigate" and visualize information once assembled.
Pulling information refers to the ability of supported analysts to have
question and answer capabilities. Starting with a known requirement,
an analyst could submit questions to a Q&A system which in turn would
"pull" the relevant information out of multiple data sources and
repositories. NSA is seeking a Q&A system that can operate
autonomously to interpret "pulled" information and provide automatic
responses back to the analysts with little additional human
intervention.
Pushing information refers to the software tools that would "blindly"
and without supervision push intelligence to analysts even if they had
not asked for the information. Research has sought to go beyond
current data mining of "structured" records deeper profiling of massive
unstructured data collections. Under the pushing information research
thrust companies have been involved in efforts to uncover previously
undetected patterns of activity from massive data sets. Software and
tools are also being developed that will provide alerts to analysts
when changes occur in newly arrived, but unanalyzed massive data
collections, such as telephone records.
The effort to navigate and visualize information seeks to develop
analytic tools that will allow agency analysts to take hundreds or even
thousands of small pieces of information and automatically create a
tailored and logical "picture" of that information. Using
visualization tools and techniques, intelligence analysts are
constantly seeking out previously unknown links and connections between
individual pieces of information.
Intelligence community efforts to process "structured" data includes
data-tagged signals intelligence (SIGINT) monitoring of telephone and
radio communications, imagery, human intelligence reporting, and
"open-source" commercial data, including news media reporting.
"Unstructured" data includes news and Internet video and audio and
document exploitation.
I could write volumes about the research efforts and the software
programs and tools used to process the mountains of information the NSA
and other agencies ingest. No doubt over the coming days and weeks,


have been able to identify in government documents relating to data
mining, link analysis, and ingestion, digestion, and distribution of
intelligence. My hope would be that other journalists and researchers
will follow the leads.
The following is a list of some 500 software tools, databases, data
mining and processing efforts contracted for, under development or in
use at the NSA and other intelligence agencies today:
Printer Friendly | Permalink |  | Top
 
DU AdBot (1000+ posts) Click to send private message to this author Click to view 
this author's profile Click to add 
this author to your buddy list Click to add 
this author to your Ignore list Wed May 01st 2024, 09:06 PM
Response to Original message
Advertisements [?]
 Top

Home » Discuss » Archives » General Discussion (1/22-2007 thru 12/14/2010) Donate to DU

Powered by DCForum+ Version 1.1 Copyright 1997-2002 DCScripts.com
Software has been extensively modified by the DU administrators


Important Notices: By participating on this discussion board, visitors agree to abide by the rules outlined on our Rules page. Messages posted on the Democratic Underground Discussion Forums are the opinions of the individuals who post them, and do not necessarily represent the opinions of Democratic Underground, LLC.

Home  |  Discussion Forums  |  Journals |  Store  |  Donate

About DU  |  Contact Us  |  Privacy Policy

Got a message for Democratic Underground? Click here to send us a message.

© 2001 - 2011 Democratic Underground, LLC