Memex 1.1

Wow! This is big news.

The Zapruder film of the Kennedy assassination; The Mansell Collection from London; Dahlstrom glass plates of New York and environs from the 1880s; and the entire works left to the collection from LIFE photographers Alfred Eisenstaedt, Gjon Mili, and Nina Leen. These are just some of the things you’ll see in Google Image Search today.

We’re excited to announce the availability of never-before-seen images from the LIFE photo archive. This effort to bring offline images online was inspired by our mission to organize all the world’s information and make it universally accessible and useful. This collection of newly-digitized images includes photos and etchings produced and owned by LIFE dating all the way back to the 1750s.

Only a very small percentage of these images have ever been published. The rest have been sitting in dusty archives in the form of negatives, slides, glass plates, etchings, and prints. We’re digitizing them so that everyone can easily experience these fascinating moments in time. Today about 20 percent of the collection is online; during the next few months, we will be adding the entire LIFE archive — about 10 million photos.

Thanks to Tony Hirst for spotting it first.

The story continues. Here’s Bill Thompson’s distinctive take on it.

As we have seen with flu trends, sometimes the “interesting” knowledge that can be extracted is well-concealed until comparisons can be made with other sources, as it was the correlation between some search terms and the real-world data that mattered.

Of course Google has not revealed which search terms it analysed because doing so would undermine the model’s effectiveness.

Unfortunately it is being equally reticent about how it has ensured that the data its uses is properly anonymised so that users cannot be identified on the basis of their queries.

A letter from the Electronic Privacy Information Center (EPIC) and Patient Privacy Rights to Google boss Eric Schmidt has not been answered, leaving those concerned with online privacy uncertain over the broader implications of the project.

But as Cade Metz points out in an insightful article in The Register, we may all be happy to know that a ‘flu outbreak is coming, but what happens when the disease involved is more life-threatening and the government asks Google for the names and IP addresses of anyone whose search terms indicate that they are infected?

It’s not that I don’t trust Google. I don’t trust any company, government department or individual without a good reason to do so.

In the case of search engines that claim to protect my privacy I want to know just how they do it and will not accept vague reassurances.

Hmmm… this Google Trends idea gets more intriguing by the minute. Rex Hughes read my comumn and pointed me at the Princeton Engineering Anomalies Research Program, which I guess was funded by You Know Who at the Pentagon. The project has closed, but here’s the blurb:

The Princeton Engineering Anomalies Research (PEAR) program, which flourished for nearly three decades under the aegis of Princeton University’s School of Engineering and Applied Science, has completed its experimental agenda of studying the interaction of human consciousness with sensitive physical devices, systems, and processes, and developing complementary theoretical models to enable better understanding of the role of consciousness in the establishment of physical reality. It has now incorporated its present and future operations into the broader venue of the International Consciousness Research Laboratories (ICRL), a 501(c)(3) organization chartered in the State of New Jersey. In this new locus and era, PEAR plans to expand its archiving, outreach, education, and entrepreneurial activities into broader technical and cultural context, maintaining its heritage of commitment to intellectual rigor and integrity, pragmatic and beneficial relevance of its techniques and insights, and sophistication of its spiritual implications.

It lives on here.

As usual, my colleague Tony Hirst has taken an idea and explored it in more depth. As we get further down this track we will, of course, come up against our old friend, mistaking-correlation-for-causation syndrome. But, hey, that some way away yet!

The toughest question a venture capitalist can ask a company seeking finance is “what happens if Google decides to move in on your turf?” (Thus far, nobody has come up with a good answer to that question, though we live in hope.) Skype, the VoIP company bought by eBay for an unconscionable sum some years ago, has just encountered that moment. Late yesterday I noticed a little red notice on the top of my Gmail screen saying “New Video Chat!” — Google chat had just been upgraded to handle video. And so, after a quick install and restart of the browser, it has. The guys who unloaded Skype onto eBay and pocketed the loot must be grinning from ear to ear today.

Following on from this post and today’s Observer column, I’ve had some feedback asking how one might go about using Google queries as forward indicators of economic developments. The answer is that I don’t know, but it probably hinges on finding the right search queries to map. Here are some experiments I’ve done.

This suggests that people weren’t really concerned about bank deposit guarantees until August 2008. I don’t believe this, so perhaps this is the wrong search term to be tracking.

This suggests that the peak of interest in house prices occurred in 2005 and is now in gentle decline. This might indicate that this search term is a token for general curiosity (”wonder why our house is worth at the moment?”) rather than alarm. It’s interesting to compare this with the chart for ‘negative equity’ below.

This has a regular annual cycle but is now clearly on the rise. Again, it’s not clear that it has much predictive power.

Speaks for itself, I think.

Ditto.

As I say, the trick would be to identify search terms which would give an indication of what people know or suspect about their organisational future.

There’s nothing terribly systematic about this — I was just trying to think of search terms that might reveals what people were thinking as they realise that they face an uncertain future.

Tony Hirst, who is the nearest thing to a wizard with Google data that I know, has some interesting things to say about the topic. He’s also knowledgeable about the limitations of the Google data.

This morning’s Observer column…

The most interesting aspect of the Google data, however, was revealed in a chart which compared flu queries with ‘objective’ data on incidence of the disease compiled by public health authorities. The chart suggests that the search data accurately reflects incidence - but is current rather than lagged. (The official statistics take about two weeks to collate.)

This suggests other possibilities - for example in macroeconomic management. Everyone I know in business has known for months that the UK is in recession, but it’s only lately that the authorities have been in a position to confirm that - because the official data always lag the current reality. So policymakers are in the situation of someone trying to drive a car which has a blacked-out windscreen. The driver’s only view of the road is a via TV monitor showing what was happening 10 seconds ago. How long would you give the driver before he hits a wall? We need to raise our game, and maybe intelligent use of the net offers us a way of doing it.

This is interesting — Google Flu Trends…

We have found a close relationship between how many people search for flu-related topics and how many people actually have flu symptoms. Of course, not every person who searches for “flu” is actually sick, but a pattern emerges when all the flu-related search queries from each state and region are added together. We compared our query counts with data from a surveillance system managed by the U.S. Centers for Disease Control and Prevention (CDC) and discovered that some search queries tend to be popular exactly when flu season is happening. By counting how often we see these search queries, we can estimate how much flu is circulating in various regions of the United States.

There’s a nice animation on the site showing how official health data lags Google searches.

The NYT has a report on this today.

Excerpt:

Tests of the new Web tool from Google.org, the company’s philanthropic unit, suggest that it may be able to detect regional outbreaks of the flu a week to 10 days before they are reported by the Centers for Disease Control and Prevention.

In early February, for example, the C.D.C. reported that the flu cases had recently spiked in the mid-Atlantic states. But Google says its search data show a spike in queries about flu symptoms two weeks before that report was released. Its new service at google.org/flutrends analyzes those searches as they come in, creating graphs and maps of the country that, ideally, will show where the flu is spreading.

The C.D.C. reports are slower because they rely on data collected and compiled from thousands of health care providers, labs and other sources. Some public health experts say the Google data could help accelerate the response of doctors, hospitals and public health officials to a nasty flu season, reducing the spread of the disease and, potentially, saving lives.

“The earlier the warning, the earlier prevention and control measures can be put in place, and this could prevent cases of influenza,” said Dr. Lyn Finelli, lead for surveillance at the influenza division of the C.D.C. From 5 to 20 percent of the nation’s population contracts the flu each year, she said, leading to roughly 36,000 deaths on average.

From a NYT Interview with Eric Schmidt, Google’s CEO.

Q. Earlier this week, Google walked away from an advertising partnership with Yahoo, after the Justice Department said it was planning to block it on antitrust grounds. Yahoo said it would have defended the deal in court and that it was disappointed you chose not to. Was Google less committed to this deal than Yahoo?

A. We were unsuccessful in convincing the Justice Department of something which we strongly feel, which is that providing better value to advertisers would have occurred by virtue of this deal. We concluded after a lot of soul-searching that it was not in our best interest to go through a lengthy and costly trial which we believe we ultimately would have won.

Q. This is the first time that regulators have gotten in the way of a Google deal. Are you concerned that, as many antitrust experts believe, this will happen more frequently now? And if so, was it a mistake for Google to propose the deal in the first place?

A. We have no regrets about attempting to do the right thing from our perspective. With change comes risks. This is a risk that we understood. Now you ask a hypothetical question, which is, Given that that event has occurred, is there another scenario? We don’t see one right now, but you never know.

Q. Will Google think differently about deals after this incident?

A. Probably not. I think that this was a unique situation.

Terrific Glenn Greenwald piece in Salon.com arguing that the modern craze for preferring to the US President as “Commander in Chief” is not only unconstitutional but dangerous.

If I could be granted one small political wish, it would be the permanent elimination of this widespread, execrable Orwellian fetish of reverently referring to the President as “our commander in chief.” And Biden’s formulation here is a particularly creepy rendition, since he’s taunting opponents of Obama that, come Tuesday, they will be forced to refer to him as “our commander in chief Barack Obama” (Sarah Palin, in the very first speech she delivered after being unveiled as the Vice Presidential candidate, said of John McCain: “that’s the kind of man I want as our commander in chief,” and she’s been delivering that same line in her stump speech ever since).

The CiC usage has been assiduously promoted by George W Bush as a way of boosting his view of untramelled presidential power (the so-called ‘unitary executive’ doctrine). After all, in the military, the CiC is someone who must be obeyed. And that’s fine in the armed forces. But the president is a civilian who happens to have been elected to the highest office in the land. His authority is constitutional, not military. If George Bush ordered me to do anything I would tell him to get stuffed — unless I worked for the executive branch of the US government (where he really is the ultimate Boss of Bosses). And so should every American citizen.