News cycle, information glut

Japan made it back onto the front page of a lot of American newspapers this week. It's not as if the unraveling nuclear disaster ever got "better" in the month since the earthquake and tsunami of 11 March. It's just that this week the officials in charge of such things raised the 'ranking' of the nuclear power plant crisis to the highest level of an international scale, on a par with the worst such crisis ever, at Chernobyl in 1986.

Slow motion disaster is background news. Escalation puts a crisis above the fold (or, in this day and age, above the scroll horizon). Were you tracking the fact that there are, very unusually, street protests in Tokyo around the nuclear crisis and the risk that last month's earthquake and tsunami have surfaced? Not a lot of media focus on that...

Libya: we in the lower reaches of North America are paying plenty of attention to Libya because the U.S. has CIA boots on the ground and both U.S. and NATO planes are flying attack missions. It's a war, and each day it becomes more clear that the U.S. is backing revolt against a merciless tyrant (we knew that going in) by a weak and disorganized opposition (optimistic strategists hoped otherwise).

Syria's got a war on too, and there the balance is tilted even more heavily toward government "security" forces in tanks against protesters who do not seem to be armed at all ... and there's not much in the way of journalistic witness either.

Egypt? Tunesia? Yemen? Conflict, betrayal, dictatorial recidivism are all still in play. But there's only so much that fits above the fold.

I've been meaning to write about Bradley Manning, the Pfc. being held in the military brig at Quantico under conditions that, "according to 250 of America's most eminent legal scholars [...] are illegal, unconstitutional, and could even amount to torture," as the U.K.'s Guardian reported over the weekend. The U.S. government's torture of Pfc. Manning -- accused of passing classified material to Wikileaks -- has been the topic of protest and many newspaper editorials. But I haven't pushed his imprisonment and treatment to the front of my to-blog-about queue because ... well ... because ... it's overwhelming how much truly awful news there is to assimilate.

And the deficit hoax? The attempt by one major U.S. party to destroy the government and the failure of the other to defend it? Not going there. Not today.

A colleague brought a Gartner article to my attention the other day: 'Big Data' Is Only the Beginning of Extreme Information Management. Here are a couple of sentences that caught my eye:

Big data has such a vast size that it exceeds the capacity of traditional data management technologies; it requires the use of new or exotic technologies simply to manage the volume alone. But processing matters, too.

The article topic is not what the context of this post might lead you to think. The Gartner authors are discussing how businesses and other organizations ought to approach management and analysis of very large data sets -- from consumer spending and web-surfing patterns, to massive sets of scientific data -- the sort of big data that is more and more regularly the basis of big business decisions.

But ... yeah.

The vast size of big data about what's happening across the globe is blowing my wetware's data management fuses, right and left. And it ain't all about taking in data. It's about processing too.

The Global Information Industry Center at UC San Diego explained in a research report:

In 2008, Americans consumed information for about 1.3 trillion hours, an average of almost 12 hours per day. Consumption totaled 3.6 zettabytes and 10,845 trillion words, corresponding to 100,500 words and 34 gigabytes for an average person on an average day. A zettabyte is 10 to the 21st power bytes, a million million gigabytes. These estimates are from an analysis of more than 20 different sources of information, from very old (newspapers and books) to very new (portable computer games, satellite radio, and Internet video). Information at work is not included.

Um. Go back and read that last sentence. That one really makes my head spin. I work in information technology. Tracking and organizing information at work is pretty much what I do. And keep in mind: the GIIC report analyzed information consumption two years ago. Nobody imagines those numbers are going down, right?

Given the 21st century information glut, how is a citizen to participate responsibly in democratic governance? Reader suggestions welcome......




Thanks to Kitamura Yayoi for bringing the Tokyo anti-nuclear street-protest to my attention; and to the Argonne National Laboratory for sharing the included photo of its IBM Blue Gene/P supercomputer on Flickr.