The predictive power of search engine queries

This morning’s Observer column

The most interesting aspect of the Google data, however, was revealed in a chart which compared flu queries with ‘objective’ data on incidence of the disease compiled by public health authorities. The chart suggests that the search data accurately reflects incidence – but is current rather than lagged. (The official statistics take about two weeks to collate.)

This suggests other possibilities – for example in macroeconomic management. Everyone I know in business has known for months that the UK is in recession, but it’s only lately that the authorities have been in a position to confirm that – because the official data always lag the current reality. So policymakers are in the situation of someone trying to drive a car which has a blacked-out windscreen. The driver’s only view of the road is a via TV monitor showing what was happening 10 seconds ago. How long would you give the driver before he hits a wall? We need to raise our game, and maybe intelligent use of the net offers us a way of doing it.

Dr Google

This is interesting — Google Flu Trends…

We have found a close relationship between how many people search for flu-related topics and how many people actually have flu symptoms. Of course, not every person who searches for “flu” is actually sick, but a pattern emerges when all the flu-related search queries from each state and region are added together. We compared our query counts with data from a surveillance system managed by the U.S. Centers for Disease Control and Prevention (CDC) and discovered that some search queries tend to be popular exactly when flu season is happening. By counting how often we see these search queries, we can estimate how much flu is circulating in various regions of the United States.

There’s a nice animation on the site showing how official health data lags Google searches.

The NYT has a report on this today.

Excerpt:

Tests of the new Web tool from Google.org, the company’s philanthropic unit, suggest that it may be able to detect regional outbreaks of the flu a week to 10 days before they are reported by the Centers for Disease Control and Prevention.

In early February, for example, the C.D.C. reported that the flu cases had recently spiked in the mid-Atlantic states. But Google says its search data show a spike in queries about flu symptoms two weeks before that report was released. Its new service at google.org/flutrends analyzes those searches as they come in, creating graphs and maps of the country that, ideally, will show where the flu is spreading.

The C.D.C. reports are slower because they rely on data collected and compiled from thousands of health care providers, labs and other sources. Some public health experts say the Google data could help accelerate the response of doctors, hospitals and public health officials to a nasty flu season, reducing the spread of the disease and, potentially, saving lives.

“The earlier the warning, the earlier prevention and control measures can be put in place, and this could prevent cases of influenza,” said Dr. Lyn Finelli, lead for surveillance at the influenza division of the C.D.C. From 5 to 20 percent of the nation’s population contracts the flu each year, she said, leading to roughly 36,000 deaths on average.

Google and the Yahoo ‘deal’ that wasn’t

From a NYT Interview with Eric Schmidt, Google’s CEO.

Q. Earlier this week, Google walked away from an advertising partnership with Yahoo, after the Justice Department said it was planning to block it on antitrust grounds. Yahoo said it would have defended the deal in court and that it was disappointed you chose not to. Was Google less committed to this deal than Yahoo?

A. We were unsuccessful in convincing the Justice Department of something which we strongly feel, which is that providing better value to advertisers would have occurred by virtue of this deal. We concluded after a lot of soul-searching that it was not in our best interest to go through a lengthy and costly trial which we believe we ultimately would have won.

Q. This is the first time that regulators have gotten in the way of a Google deal. Are you concerned that, as many antitrust experts believe, this will happen more frequently now? And if so, was it a mistake for Google to propose the deal in the first place?

A. We have no regrets about attempting to do the right thing from our perspective. With change comes risks. This is a risk that we understood. Now you ask a hypothetical question, which is, Given that that event has occurred, is there another scenario? We don’t see one right now, but you never know.

Q. Will Google think differently about deals after this incident?

A. Probably not. I think that this was a unique situation.

Drop the CiC cliche

Terrific Glenn Greenwald piece in Salon.com arguing that the modern craze for preferring to the US President as “Commander in Chief” is not only unconstitutional but dangerous.

If I could be granted one small political wish, it would be the permanent elimination of this widespread, execrable Orwellian fetish of reverently referring to the President as “our commander in chief.” And Biden’s formulation here is a particularly creepy rendition, since he’s taunting opponents of Obama that, come Tuesday, they will be forced to refer to him as “our commander in chief Barack Obama” (Sarah Palin, in the very first speech she delivered after being unveiled as the Vice Presidential candidate, said of John McCain: “that’s the kind of man I want as our commander in chief,” and she’s been delivering that same line in her stump speech ever since).

The CiC usage has been assiduously promoted by George W Bush as a way of boosting his view of untramelled presidential power (the so-called ‘unitary executive’ doctrine). After all, in the military, the CiC is someone who must be obeyed. And that’s fine in the armed forces. But the president is a civilian who happens to have been elected to the highest office in the land. His authority is constitutional, not military. If George Bush ordered me to do anything I would tell him to get stuffed — unless I worked for the executive branch of the US government (where he really is the ultimate Boss of Bosses). And so should every American citizen.

Google pays peanuts for pole position

This morning’s Observer column

Is there such a thing as a ‘win-win’ situation? Journalistic cynicism says no. What the phrase usually means is that some people get more than they deserve and others get less – but not so little that they scream blue murder. The big puzzle about the ‘ground-breaking settlement’ announced last week between Google and its legal opponents, the Authors Guild and the Association of American Publishers, is whether it really is – as all parties claim – a victory for everyone…

The Bookseller magazine picked up on this piece and posted a good summary.

The G-phone: first review

David Pogue has had a good look at the first G-phone to roll off the line. It’s a useful review. His conclusion:

So there’s your G1 report card: software, A-. Phone, B-. Network, C.

So here’s what will happen. 1. The software (done by Google) will improve rapidly. 2. Phone manufacturers will eventually produce a suitable handset. 3. The phone will be available on all networks in due course. All this will take a while, so my hunch is that the iPhone has a clear run for the time being.

I’m still tempted to try a G-phone when it arrives in the UK next month, though.

The aphrodisiac effect of power

This morning’s Observer column

Once upon a time, the ultimate put-down to a bright spark was to say, ‘well, if you’re so smart, how come you’re not so rich?’. Wall Street Crash 2.0 has rather undermined this ploy, by making it clear that an awful lot of very rich folks were anything but smart. It turns out that we were unduly dazzled by the Masters of the Universe, but we had to wait until they had vaporised the US economy before getting wise to the fact.

Actually, this was just a special case of a more general human weakness – our tendency to lose all capacity for critical thought when confronted by great wealth or power. This ‘aphrodisiac effect’ seems to be ubiquitous. One saw it, for example, in the way leggy females used to throw themselves at Henry Kissinger, a stumpy troglodyte who just happened to be US Secretary of State. And we see it in the way even hardened hacks go weak when offered an audience with Bill Gates, Warren Buffett or even, God help us, Steve Ballmer, chief of Microsoft…

Androids and walled gardens

This morning’s Observer column

‘We are all,’ said Keynes, ‘the slaves of some defunct philosopher.’ The question that will increasingly preoccupy mobile-phone executives from now on is which deceased sage is more appropriate for their product. Up to now, the relevant thinker has been Lenin – who, you may remember, was a control freak. Given that most mobile operators had their origins in traditional telephone companies – which liked to think they ‘owned’ their customers – this is hardly surprising. These outfits have control freakery in their corporate DNA.

Last week, the first mobile phone based on Google’s Android operating system was released by T-Mobile in the US. (The network is bringing it to the UK in November.) The philosophy underpinning the device is radically different from anything we have seen thus far in the mobile-phone market. The world is about to become a more interesting place. And what happens next could have far-reaching implications…

CORRECTION: An observant reader, Duncan Thomas, has just spotted an error in the piece as published. The piece says that “the most important difference [between the Google phone and the iPhone] is that the Android software ecosystem will not be an uncontrolled, open space”. That ‘not’ ought to have been deleted. Drat and double drat.

LATER: Webmonkey’s five reasons why Android might do the business

1. It promises to run on most modern smart phones – More cell networks will support Android than iPhone does — the iPhone is bound to just AT&T. Mobile providers NTT DoCoMo, Sprint Nextel, T-Mobile and more have committed to the project. Also, more handsets will operate on it. You might even get more life out of your old phone if it supports it. Handset manufactures HTC, LG, Motorola and Samsung have already signed on.
2. It’s open-source software – Any programmer can whip up some code to match popular features from any other phone. Under the Apache license, any programmer can take the code and port their own version of the OS.
3. It has support for Google products out of the box – The latest Android demonstration displayed the phone’s compass prominently in Google Maps. You can bet Google will have the latest and greatest features of their software running on Android before it hits other operators.
4. Third-party developers have more access – iPhone prohibits people from using its internet capabilities for things like VoIP or an alternative browser. Android’s API allows you to create an application for anything, even the dialing software. The evidence is in the 50 applications already developed for the Android Developer Challenge last May.
5. Android allows for ‘unlocked’ phones – Most handsets in America, including the iPhone, are locked by software to a cell phone provider’s network. While there are various ways to jailbreak, it’s not easy and might break your terms of service. The availability of downloading and installing your own unlocked OS might just change the game in respect to shopping for mobile phone providers and signing contracts. If this method gets more popular, it is conceivable phone networks may drop the contracts in lieu of (better) European pre-pay pricing.

Le G-phone est arrivé

Good Morning Silicon Valley is spot on. It’s not about phones, it’s about philosophies.

It comes down to closed vs. open. In political terms, the Apple environment is like Singapore, where some freedoms may be ceded in favor of providing a pleasant and orderly experience, and Google, with its Android platform, is like a loud and messy New England town meeting. Apple has one iPhone, a tightly controlled App Store for third-party programs, and a touchscreen design that favors consumption of iTunes entertainment. The G1 is but the first of many Android-based devices to come, all of which will be served by the wide-open Android Market, and its design, featuring a real keyboard, leans toward typing-oriented functions like mail, messaging and mobile search, not coincidentally all Google strong suits. If you’re already happy in the Apple ecosystem, or with an “it just works (most of the time)” approach to tech in general, and you’re in the smart-phone market, there’s probably not much that Android handset manufacturers can come up with that will tempt you away from the iPhone. If you’re already happy in the Google ecosystem, then the tight integration of Google applications and services and the breadth of third-party development possibilities will make an Android-based phone more appealing. At the core, the iPhone and the Android phones may not really be the direct competitors they’re made out to be, but rather comparable alternatives whose appeal depends mostly on whether your tastes and needs put you in the closed or open camp.

Walt Mossberg’s useful first impressions are here.

Meanwhile, Google has been posting demo videos like this on YouTube.