NYT: A small group of National Security Agency officials slipped into Silicon Valley on one of the agency's periodic technology shopping expeditions this month.
On the wish list, according to several venture capitalists who met with the officials, were an array of technologies that underlie the fierce debate over the Bush administration's anti-terrorist eavesdropping program: computerized systems that reveal connections between seemingly innocuous and unrelated pieces of information.
The tools they were looking for are new, but their application would fall under the well-established practice of data mining: using mathematical and statistical techniques to scan for hidden relationships in streams of digital data or large databases.
Supercomputer companies looking for commercial markets have used the practice for decades. Now intelligence agencies, hardly newcomers to data mining, are using new technologies to take the practice to another level. (...)
Much of the recent work on data mining has been aimed at even more sophisticated applications. The National Security Agency has invested billions in computerized tools for monitoring phone calls around the world — not only logging them, but also determining content — and more recently in trying to design digital vacuum cleaners to sweep up information from the Internet.
Last September, the N.S.A. was granted a patent for a technique that could be used to determine the physical location of an Internet address — another potential category of data to be mined. The technique, which exploits the tiny time delays in the transmission of Internet data, suggests the agency's interest in sophisticated surveillance tasks like trying to determine where a message sent from an Internet address in a cybercafe might have originated.
An earlier N.S.A. patent, in 1999, focused on a software solution for generating a list of topics from computer-generated text. Such a capacity hints at the ability to extract the content of telephone conversations automatically. That might permit the agency to mine millions of phone conversations and then select a handful for human inspection.
As the N.S.A. visit to the Silicon Valley venture capitalists this month indicates, the actual development of such technologies often comes from private companies.
In 2003, Virage, a Silicon Valley company, began supplying a voice transcription product that recognized and logged the text of television programming for government and commercial customers. Under perfect conditions, the system could be 95 percent accurate in capturing spoken text. Such technology has potential applications in monitoring phone conversations as well...(more)