Can Google Flu Trends Be Manipulated?

Posted on by

Interesting contribution to the discussion by Ed Felten.

My concern today is whether Flu Trends can be manipulated. The system makes inferences from how people search, but people can change their search behavior. What if a person or a small group set out to convince Flu Trends that there was a flu outbreak this week?

An obvious approach would be for the conspirators to do lots of searches for likely flu-related terms, to inflate the count of flu-related searches. If all the searches came from a few computers, Flu Trends could presumably detect the anomalous pattern and the algorithm could reduce the influence of these few computers. Perhaps this is already being done; but I don’t think the research paper mentions it.

A more effective approach to spoofing Flu Trends would be to use a botnet — a large collection of hijacked computers — to send flu-related searches to Google from a larger number of computers. If the added searches were diffuse and well-randomized, they would be very hard to distinguish from legitimate searches, and the Flu Trends would probably be fooled.

This possibility is not discussed in the Flu Trends research paper. The paper conspicuously fails to identify any of the search terms that the system is looking for. Normally a paper would list the terms, or at least give examples, but none of the terms appear in the paper, and the Flu Trends web site gives only “flu” as an example search term. They might be withholding the search terms to make manipulation harder, but more likely they’re withholding the search terms for business reasons, perhaps because the terms have value in placing or selling ads.

Cyberchondria

Posted on by

Wow! Microsoft Research has just published a research study on what happens when people seek health information on the Web. Abstract:

The World Wide Web provides an abundant source of medical information. This information can assist people who are not healthcare professionals to better understand health and disease, and to provide them with feasible explanations for symptoms. However, the Web has the potential to increase the anxieties of people who have little or no medical training, especially when Web search is employed as a diagnostic procedure. We use the term cyberchondria to refer to the unfounded escalation of concerns about common symptomatology, based on the review of search results and literature on the Web. We performed a large-scale, longitudinal, log-based study of how people search for medical information online, supported by a large-scale survey of 515 individuals’ health-related search experiences. We focused on the extent to which common, likely innocuous symptoms can escalate into the review of content on serious, rare conditions that are linked to the common symptoms. Our results show that Web search engines have the potential to escalate medical concerns. We show that escalation is influenced by the amount and distribution of medical content viewed by users, the presence of escalatory terminology in pages visited, and a user’s predisposition to escalate versus to seek more reasonable explanations for ailments. We also demonstrate the persistence of post-session anxiety following escalations and the effect that such anxieties can have on interrupting user’s activities across multiple sessions. Our findings underscore the potential costs and challenges of cyberchondria and suggest actionable design implications that hold opportunity for improving the search and navigation experience for people turning to the Web to interpret common symptoms.

The Obama rebound?

Posted on by

Sometimes, it’s ifficult to know what to think. This morning the Today programme carried a chilling article about the upsurge in US gun sales since Obama’s election. On the other hand, here’s an interesting reflection Mark Anderson.

The old adage says, The darkest hour is just before the dawn.

It’s hard to be upbeat these days, when every statistic is worse than the last. But the other day, as I was considering predictions for the coming year, a thought occurred to me: we are experiencing the waning days of the administration I have repeatedly called the worst in US history. Of course things look dark.

Is it possible, once the new administration is in place, that hearing daily announcements of LIPs (leadership, ideas and plans) put forward by people who are both smart and qualified, will have the opposite effect on the public from the constant drumming of fear we continue to have today?

Of course.

Is it also the case that markets react more to perception than to ground truth?

Generally, yes.

So I asked myself, what will the state of mind be of the average American, say, three weeks into the next administration – let’s say, by Valentine’s Day, February 14th?

If their house has just been foreclosed and their car repossessed, we know what they’ll be thinking. But otherwise, I expect it will be radically more optimistic than it is today.

Is that enough to provide a market rebound? It could be.

The future of news

Posted on by

Interesting long post by Jeff Jarvis…

It’s fair to expect me to put forward scenarios for the future of news. In a sense, that’s all I ever do here, but there’s no one permalink summarizing my apparently endless prognostication. So here is a snapshot of – a strawman for – where I think particularly local news might go. What follows is just a long – I’m sorry – summary of what I’ve written here over time and an extension of the one model I think we need to expand coming out of the conference, where one lesson I took away is that news – on both the content and business side – will no longer be controlled by a single company but will be collaborative…

Those Obama appointments

Posted on by

From David Brooks

Jan. 20, 2009, will be a historic day. Barack Obama (Columbia, Harvard Law) will take the oath of office as his wife, Michelle (Princeton, Harvard Law), looks on proudly. Nearby, his foreign policy advisers will stand beaming, including perhaps Hillary Clinton (Wellesley, Yale Law), Jim Steinberg (Harvard, Yale Law) and Susan Rice (Stanford, Oxford D. Phil.).

The domestic policy team will be there, too, including Jason Furman (Harvard, Harvard Ph.D.), Austan Goolsbee (Yale, M.I.T. Ph.D.), Blair Levin (Yale, Yale Law), Peter Orszag (Princeton, London School of Economics Ph.D.) and, of course, the White House Counsel Greg Craig (Harvard, Yale Law).

This truly will be an administration that looks like America, or at least that slice of America that got double 800s on their SATs. Even more than past administrations, this will be a valedictocracy — rule by those who graduate first in their high school classes. If a foreign enemy attacks the United States during the Harvard-Yale game any time over the next four years, we’re screwed.

Already the culture of the Obama administration is coming into focus. Its members are twice as smart as the poor reporters who have to cover them, three times if you include the columnists. They typically served in the Clinton administration and then, like Cincinnatus, retreated to the comforts of private life — that is, if Cincinnatus had worked at Goldman Sachs, Williams & Connolly or the Brookings Institution. So many of them send their kids to Georgetown Day School, the posh leftish private school in D.C., that they’ll be able to hold White House staff meetings in the carpool line…

Lovely piece which, in the end, is not quite as cynical as its opening paras might suggest.

Obama’s energy plan

Posted on by

From the Obama/Biden site…

# Provide short-term relief to American families facing pain at the pump
# Help create five million new jobs by strategically investing $150 billion over the next ten years to catalyze private efforts to build a clean energy future.
# Within 10 years save more oil than we currently import from the Middle East and Venezuela combined.
# Put 1 million Plug-In Hybrid cars — cars that can get up to 150 miles per gallon — on the road by 2015, cars that we will work to make sure are built here in America.
# Ensure 10 percent of our electricity comes from renewable sources by 2012, and 25 percent by 2025.
# Implement an economy-wide cap-and-trade program to reduce greenhouse gas emissions 80 percent by 2050.

Thanks to Brian for the link.

Eh?

Posted on by

From today’s NYTimes.com

It’s a long way from $700 billion, but the media start-up Six Apart is introducing its own economic bailout plan.

The TypePad Journalist Bailout Program offers recently terminated bloggers and journalists a free pro account (worth $150 annually) on the company’s popular blogging platform. In addition to the free yearly membership, the 20 to 30 journalists who are accepted will receive professional tech support, placement on the company’s blog aggregation site, Blogs.com, and automatic enrollment in the company’s advertising revenue-sharing program.

Anil Dash, a former blogger and current vice president at Six Apart, announced the program Nov. 14, shortly after the company made its own staff cuts. Mr. Dash fired off a blog post: “Hello, recently-laid-off or fearful-of-layoffs journalist! We’re Six Apart (you know us as the nice folks who make Movable Type or TypePad, which maybe you used for blogging at your old newspaper or magazine) and we want to help you.”

On Monday morning, he had roughly 50 e-mail applications in his inbox, and they have continued to pour in, totaling nearly 300 so far. “It was a bit of a surprise how quickly word got out,” Mr. Dash said. “This has struck a nerve.”

I don’t get this. What’s to stop these people getting a free Blogger or WordPress account? Is this another example of print journalists being clueless about the Web? Am I missing something?

Barry O’Bama

Posted on by

Needless to say, Barack has Irish ancestors. (Doesn’t everybody famous?) His late mother, Anne Dunham, was a descendant of Fulmouth Kearney who left Moneygall, Co. Offaly (Latitude: 52° 52′ 49 N, Longitude: 7° 57 ‘ 31 W) , for the US in 1850.

But here’s the neat twist: Mr Kearney was a Protestant! Or, as my dear mother used to say (for she was not a woman to mince her words and took a dim view of non-Catholics), a ‘Black Proteshtant’. Yippee!