Wolfram|Alpha

Been playing with Wolfram|Alpha today:

Wolfram|Alpha's long-term goal is to make all systematic knowledge immediately computable and accessible to everyone. We aim to collect and curate all objective data; implement every known model, method, and algorithm; and make it possible to compute whatever can be computed about anything. Our goal is to build on the achievements of science and other systematizations of knowledge to provide a single source that can be relied on by everyone for definitive answers to factual queries.

Wolfram|Alpha aims to bring expert-level knowledge and capabilities to the broadest possible range of people—spanning all professions and education levels. Our goal is to accept completely free-form input, and to serve as a knowledge engine that generates powerful results and presents them with maximum clarity.

Wolfram|Alpha is an ambitious, long-term intellectual endeavor that we intend will deliver increasing capabilities over the years and decades to come. With a world-class team and participation from top outside experts in countless fields, our goal is to create something that will stand as a major milestone of 21st century intellectual achievement.

Information overload: it's official

We’ve hit information overload: The amount of data people create now exceeds the amount of space available to store them.

People sent emails, took digital pictures, processed credit cards and generally did things that collectively created 281 exabytes of data by the end 2007, according to the research company IDC. (“Exabyte” sounds made up, but it’s a real term meaning 1,000,000,000,000 megabytes.) IDC also added up all the computer drives, backup tapes, CDs, DVDs, memory sticks and other devices that store data and estimated that their total capacity is only 264 exabtyes.

There aren’t data just drifting in the ether somewhere. A lot of the data that get created—say, an Internet phone call–never get stored. Other data get erased or recorded over. It’s the digital equivalent of a conversation going in one ear and out the other. But for the first time in human history, we couldn’t save all this information if we wanted to, according to IDC.

It's Official: There's Too Much Information (Wall Street Journal Business Technology Blog)

Learn it well

If you're going to be a high achiever, you're going to be in lots of situations where you're going to be quickly making decisions in the presence of incomplete or incorrect information, under intense time pressure, and often under intense political pressure. You're going to screw up -- frequently -- and the screwups will have serious consequences, and you'll feel incredibly stupid every time. It can't faze you -- you have to be able to just get right back up and keep on going.

That may be the most valuable skill you can ever learn. Make sure you start learning it early.

The Pmarca Guide to Career Planning, part 2: Skills and education (Marc Andreesen)

Perils of KM

I think it's safe to say that, in my 20+ years in Information Technology, I have been involved in my share of Knowledge Management (KM) initiatives.

(For the uninitiated, "knowledge management" is the capture, collection, organization, and storage of information within an organization, together with a mechanism for allowing this information to be shared easily. And yes, the World Wide Web can be viewed as a giant, collective experiment in KM, if you'd like to think of it that way; a more precise example would be something like Wikipedia.)

KM, as practiced in the current day, is an ambitious and very necessary effort to tackle a very real problem: how do you get key information to the people who need it?

But KM efforts fall down, again and again, because of this simple but universal truth:

Machines are Dumb.

And for your KM effort to be successful, you need someone Smart (usually several someones, a whole team of human editors) reading Every. Single. Thing. that goes into your KM database, keeping it current and accurate, and making sure that bad or undesirable information is not captured.

Otherwise, you wind up with situations like this:

Last March, the federal government set up a Web site to make public a vast archive of Iraqi documents captured during the war. The Bush administration did so under pressure from Congressional Republicans who had said they hoped to “leverage the Internet” to find new evidence of the prewar dangers posed by Saddam Hussein.

But in recent weeks, the site has posted some documents that weapons experts say are a danger themselves: detailed accounts of Iraq’s secret nuclear research before the 1991 Persian Gulf war. The documents, the experts say, constitute a basic guide to building an atom bomb.

Last night, the government shut down the Web site after The New York Times asked about complaints from weapons experts and arms-control officials. A spokesman for the director of national intelligence said access to the site had been suspended “pending a review to ensure its content is appropriate for public viewing.”

Officials of the International Atomic Energy Agency, fearing that the information could help states like Iran develop nuclear arms, had privately protested last week to the American ambassador to the agency, according to European diplomats who spoke on condition of anonymity because of the issue’s sensitivity. One diplomat said the agency’s technical experts “were shocked” at the public disclosures.

U.S. Web Archive Is Said to Reveal a Nuclear Primer (New York Times, November 3, 2006)

This is, of course, the kind of risk you run when you publicly post tens of thousands of documents in a language (Arabic) that the vast majority of your knowledge managers do not speak or read.

The irony is--if I have understood the news story correctly--that the documents that caused the current furor were UN reports that were written and distributed in English.

Ouch.

Update (and bump), 11/4: Chap adds some interesting observations over at his place.

Intelligence community goes wiki

The U.S. intelligence community on Tuesday unveiled its own secretive version of Wikipedia, saying the popular online encyclopedia format known for its openness is key to the future of American espionage.

The office of U.S. intelligence czar John Negroponte announced Intellipedia, which allows intelligence analysts and other officials to collaboratively add and edit content on the government's classified Intelink Web much like its more famous namesake on the World Wide Web.

A "top secret" Intellipedia system, currently available to the 16 agencies that make up the U.S. intelligence community, has grown to more than 28,000 pages and 3,600 registered users since its introduction on April 17. Less restrictive versions exist for "secret" and "sensitive but unclassified" material.

U.S. intelligence unveils spy version of Wikipedia - Yahoo! News