Computerised hostility detection

Fascinating article in today’s New York Times

WASHINGTON, Oct. 3 — A consortium of major universities, using Homeland Security Department money, is developing software that would let the government monitor negative opinions of the United States or its leaders in newspapers and other publications overseas.

Such a “sentiment analysis” is intended to identify potential threats to the nation, security officials said.

Researchers at institutions including Cornell, the University of Pittsburgh and the University of Utah intend to test the system on hundreds of articles published in 2001 and 2002 on topics like President Bush’s use of the term “axis of evil,” the handling of detainees at Guantánamo Bay, the debate over global warming and the coup attempt against President Hugo Chávez of Venezuela.

A $2.4 million grant will finance the research over three years.

American officials have long relied on newspapers and other news sources to track events and opinions here and abroad, a goal that has included the routine translation of articles from many foreign publications and news services.

The new software would allow much more rapid and comprehensive monitoring of the global news media, as the Homeland Security Department and, perhaps, intelligence agencies look “to identify common patterns from numerous sources of information which might be indicative of potential threats to the nation,” a statement by the department said.

It could take several years for such a monitoring system to be in place, said Joe Kielman, coordinator of the research effort. The monitoring would not extend to United States news, Mr. Kielman said.

“We want to understand the rhetoric that is being published and how intense it is, such as the difference between dislike and excoriate,” he said.

Even the basic research has raised concern among journalism advocates and privacy groups, as well as representatives of the foreign news media.

“It is just creepy and Orwellian,” said Lucy Dalglish, a lawyer and former editor who is executive director of the Reporters Committee for Freedom of the Press.

Andrei Sitov, Washington bureau chief of the Itar-Tass news agency of Russia, said he hoped that the objective did not go beyond simply identifying threats to efforts to stifle criticism about an American president or administration.

“This is what makes your country great, the open society where people can criticize their own government,” Mr. Sitov said…

BTW: “Sentiment analysis” is what this stuff is called in the comp.sci community.