The predictive power of search engine queries

This morning’s Observer column

The most interesting aspect of the Google data, however, was revealed in a chart which compared flu queries with ‘objective’ data on incidence of the disease compiled by public health authorities. The chart suggests that the search data accurately reflects incidence – but is current rather than lagged. (The official statistics take about two weeks to collate.)

This suggests other possibilities – for example in macroeconomic management. Everyone I know in business has known for months that the UK is in recession, but it’s only lately that the authorities have been in a position to confirm that – because the official data always lag the current reality. So policymakers are in the situation of someone trying to drive a car which has a blacked-out windscreen. The driver’s only view of the road is a via TV monitor showing what was happening 10 seconds ago. How long would you give the driver before he hits a wall? We need to raise our game, and maybe intelligent use of the net offers us a way of doing it.

Depth of field

One of the delights of the Nikon D700 is that all my old Nikon glassware has come back into its own. This was taken earlier this evening with the ancient 50mm f1.2 lens at full aperture.

Leaving the Ivory Tower

Tony Hirst pointed me at this sombre piece by an academic (with tenure) who’s leaving HE because he’s pissed off with the way things are going. Excerpt;

After too many years at this job (I am in my mid-40s), I have grown to question higher education in ways that cannot be rectified by a new syllabus, or a sabbatical, or, heaven forbid, a conference roundtable. No, my troubles with this treasured profession are both broad and deep, and they begin with a fervent belief that most of today’s college students, especially those that come to college straight from high school, are unnecessarily coddled. Professors and administrators seek to “nurture” and “engage” and they are doing so at the expense of teaching. The result: a discernable and precipitous decline in the quality of college students. More of them come to campus with dreadful study habits. Too few of them read for pleasure. Too many drink and smoke excessively. They are terribly ill-prepared for four years of hard work, and most dangerously, they do not think that college should be arduous. Instead they perceive college as an overnight recreation center in which they exercise, eat, and in between playing extracurricular sports, they carry books around. If a professor is lucky, the books are being skimmed hours before class.

How do I know that my concerns are not unique to my employer, or my classroom? My students are brutally honest – they tell me with candor and without shame that their peers think of college as a four year cruise without a destination.

No doubt these students deserve some blame for their lethargy, but some culpability lies with their professors, and the administrators who ostensibly but unsuccessfully provide vision and direction. Today’s faculty and administrators capitulate to students’ demands in innumerable ways. They hold classes outside on sunny days, not really caring if there is no blackboard, or if the students are admiring each other instead of the texts to be dissected. They encourage students to think of college as a “comfortable” and “supportive” community, not as a means to acquire necessary skills. Far too many of my colleagues are dialing in – showing up late, popping in videos during class, assigning group projects, or sitting in a circle and asking students how they feel. Why they have abandoned classroom rigor is something that only they can answer. But one answer is simple – students flock to these popular classes, probably because they cater to the students’ worst sensibilities. Homework is minimal, or sometimes optional. Surprise quizzes are considered unfair. Late assignments are not failed. Some grades are even negotiable…

I know lots of academics in various institutions who feel like this. But I guess most of them won’t quit: “better to hold on to nurse”, as Harold Macmillan used to say, “for fear of something worse”.

LATER: Nice email from Roger Whittaker who says: “I think Macmillan was quoting Belloc – one of the Cautionary Tales“.

Net Gen (contd)

The Economist has a glowing review of Don Tapscott’s new book.

In the past two years, Don Tapscott has overseen a $4.5m study of nearly 8,000 people in 12 countries born between 1978 and 1994. In “Grown Up Digital” he uses the results to paint a portrait of this generation that is entertaining, optimistic and convincing. The problem, he suspects, is not the net generation but befuddled baby-boomers, who once sang along with Bob Dylan that “something is happening here, but you don’t know what it is”, yet now find that they are clueless about the revolutionary changes taking place among the young.

“As the first global generation ever, the Net Geners are smarter, quicker and more tolerant of diversity than their predecessors,” Mr Tapscott argues. “These empowered young people are beginning to transform every institution of modern life.” They care strongly about justice, and are actively trying to improve society—witness their role in the recent Obama campaign, in which they organised themselves through the internet and mobile phones and campaigned on YouTube. Mr Tapscott’s prescient chapter on “The Net Generation and Democracy: Obama, Social Networks and Citizen Engagement” alone should ensure his book a wide readership…

W: an empty shell?

Last night we went to see W, Oliver Stone’s biopic about the life and disasters of George W Bush. The title comes from the fact that he is apparently known as ‘doubleya’ (or even ‘dubya’) by his friends and most of his family (though his Pa calls him ‘Junior’). I was disposed to like the film, but I’m afraid it’s a turkey despite a few good performances (notably by Josh Brolin as Dubya, Elizabeth Banks as Laura Bush and Richard Dreyfuss as an authentically creepy Dick Cheney). I’m glad I saw it after Obama’s victory rather than before the election, because the main thought it provokes is wonderment that such a gang of warped, dysfunctional cretins could take over the government of a civilised country.

The obvious comparison is with Stone’s Nixon, which I think is a fine and absorbing film. One reason why W is so unsatisfactory might be that the current President’s character isn’t able to bear the weight, whereas Nixon, for all his many defects (or perhaps because of them) was a genuinely interesting and complex man. So in making W Stone was effectively probing an empty shell.

All of which makes me think that perhaps the upcoming Frost/Nixon film might be genuinely interesting.

Palin: English as she is spooken

Nice column by Dick Cavett …

What on earth are our underpaid teachers, laboring in the vineyards of education, supposed to tell students about the following sentence, committed by the syntax-serial-killer from Wasilla High and gleaned by my colleague Maureen Dowd for preservation for those who ask, “How was it she talked?”

“My concern has been the atrocities there in Darfur and the relevance to me with that issue as we spoke about Africa and some of the countries there that were kind of the people succumbing to the dictators and the corruption of some collapsed governments on the continent, the relevance was Alaska’s investment in Darfur with some of our permanent fund dollars.”

And, she concluded, “never, ever did I talk about, well, gee, is it a country or a continent, I just don’t know about this issue.”

It’s admittedly a rare gift to produce a paragraph in which whole clumps of words could be removed without noticeably affecting the sense, if any.

(A cynic might wonder if Wasilla High School’s English and geography departments are draped in black.)

Reykjavik-on-Thames?

I know nothing about economics and so tended to regard my suspicions about the possibility of national bankruptcy as the fears of an uninformed mind. But Willem Buiter is a highly-informed mind, and he has a long, thoughtful blog post today on the FT site in which he discusses the possibility of a UK default.

The key question is, can the government meet all these fiscal commitments, whether firm or flaccid, unconditional or contingent and explicit or implicit ? Does it have the resources, now and in the future, to issue the additional debt required to meet the growing volume of up-front obligations it has taken on?

It’s a long, closely-argued, data-laden piece, and his answer, roughly stated, is that it depends on a lot of factors. But the interesting thing is that a serious academic economist is treating a question hitherto dismissed (e.g. by Charles Goodhart) as ‘inconceivable’ as, well, conceivable.

Fruitcakes of the world, unite!

The Economist has received some hilarious objections to its endorsement of Barack Obama. This is the wackiest (from some guy in Missouri).

SIR –America’s election laws prohibit foreigners from contributing to the campaigns of elected officials. By publishing your endorsement before the election, you attempted to influence the electorate in a way that has far more impact than contributing money. You have, in effect, violated the spirit and intent of American law. Your European welfare-state mentality inevitably biased your conclusions. Americans are a centre-right people, whereas Britain is at best left-centre (word order is paramount here).

The nicest letter came from someone in Italy:

SIR – I would like to congratulate Mr Obama on his brilliant victory. In his official capacity as president of the United States he will probably have to meet our prime minister, Silvio Berlusconi. I apologise in advance.

Obama’s economics transition team

Wonderful rant by Willem Buiter in his excoriation of Obama’s economics advisory board. In a nutshell, the team is too old, has too few professional economists, too many people associated with past failures, is stuffed with protectionists — and has too many lawyers. It’s this last that gets Professor Buiter really riled.

According to Legal Reform Now! there are 1,143,358 lawyers in the US, one for every 200 adults. The main problem is not that there are over a million socially unproductive lawyers in the US. The problem is that these lawyers are an essential component of a dysfunctional legal framework that has created the most litigious society in the world. The damage this dysfunctional legal framework causes must be measured not primarily by the direct cost of litigation, astounding though it is, but through the actions not undertaken and the creative and productive deeds not done because of fear of litigation. The first thing we do…

Except for a depressingly small minority among them, lawyers know nothing. They are incapable of logic. They don’t know the difference between necessary and sufficient conditions or between type I and type II errors. Indeed, any concept of probability is alien to them. They don’t understand the concepts of opportunity cost and trade off. They cannot distinguish between normative and positive statements. They are so focused on winning an argument through technicalities, that they no longer would recognise the truth if it bit them in the butt. If you are very lucky, a lawyer will give you nothing but the truth. You will never get the truth, let alone the whole truth. Things have degenerated to the point that lawyers and the legal profession not only routinely undermine justice, but even the law.

But the American political system is completely dominated by this largely socially unproductive and parasitic profession. Consider the membership of the House and the Senate (according to the Congressional Research Service 170 members of the House (out of 435) and 60 Senators (out of 100) are lawyers). Consider the professional training and background of past and future presidents (including Obama, 26 out of 44 presidents were lawyers) – and weep.