Algorithmic ‘integrity’

From Rough Type: Nicholas Carr’s Blog

Last week, CNET’s Elinor Mills reported on how a web search for “Martin Luther King” returns, as its first result on Google and as its second result on Windows Live Search, a web site (martinlutherking.org) operated by a white supremacist organization named Stormfront. The site, titled “Martin Luther King Jr.: A True Historical Examination,” refers to King as “The Beast” and says he was “just a sexual degenerate, an America-hating Communist, and a criminal betrayer of even the interests of his own people.” The site also features an essay on “Jews & Civil Rights” by former Ku Klux Klan official David Duke.

What’s remarkable, though, is not that a search algorithm might be gamed by extremists but that the owners of the algorithm might themselves defend the offensive result – and reject any attempt to override it as an assault on the “integrity” of their system….

Carr goes on to quote Google’s response to the CNET story:

At Google, a Web site’s ranking is determined by computer algorithms using thousands of factors to calculate a page’s relevance to any given query, a company representative said. The company can’t tweak the results because of that automation and the need to maintain the integrity of the results, she said. “In this particular example, the page is relevant to the query and many people have linked to it, giving it more PageRank than some of the other pages. These two factors contribute to its ranking.”

Microsoft’s response was even more robust:

The results on Microsoft’s search engine are “not an endorsement, in any way, of the viewpoints held by the owners of that content,” said Justin Osmer, senior product manager for Windows Live Search. “The ranking of our results is done in an automated manner through our algorithm which can sometimes lead to unexpected results,” he said. “We always work to maintain the integrity of our results to ensure that they are not editorialized.”

To which Carr tartly responds:

By “editorialized” he seems to mean “subjected to the exercise of human judgment.” And human judgment, it seems, is an unfit substitute for the mindless, automated calculations of an algorithm. We are not worthy to question the machine we have made. It is so pure that even its corruption is a sign of its integrity.

The pleasures of Amazon

When I got back from work today, the postman had delivered two delights. One is this collection of David Remnick’s journalism. He’s such a graceful writer — the kind that leaves you staring at a sentence and wondering how anything can be so succinctly elegant.

The other delight is Bob Dylan’s new album, Modern Times. It’s astonishingly original and fresh, with some tracks (Thunder on the Mountain, The Levee’s Gonna Break) which remind one of why rock music changed the world.

Press the Space bar

This is from a series of images provided by NASA of the latest Shuttle mission. The laptop on the left looks like an IBM Thinkpad. Hmmm… wonder if it’s running XP.

Clinton goes Open Source?

Steve Bell’s cartoon in the Guardian of September 28. Note top right-hand corner. What can this mean?

Later… Mystery solved. James M, from whom nothing is hidden, writes:

In his speech to the Labour Party BrownNoseFest [Clinton] introduced the word Ubuntu as expressive the interelationships of people and their circumstances in South Africa. He also mentioned the software connection.

Scary movie 10,000

The kids and I went to see Al Gore’s film, An Inconvenient Truth this evening. It was — as one review put it — “a deluxe filmed version of the ‘slide show’ Gore has been presenting and refining since 1978, in which he concisely lays out the case that our carbon-dioxide emissions trapped in the Earth’s atmosphere are systematically destroying the environment”. I had seen a version of the slide show some time ago, so the content of the film was not a surprise to me — though it was eye-opening for the kids. I came away with a few thoughts:

  • Admiration for the presentation skills Gore has refined. As someone who has to do a lot of presentations (and sometimes even has to use PowerPoint), I know how difficult it is to get across complex ideas. Larry Lessig is the best presenter I know in this respect. Gore comes pretty close.
  • Nice to see that he uses an Apple PowerBook and Keynote.
  • The ‘personal’ bits in the film — digressions about the accident that nearly killed his six-year-old son, and the lung cancer that killed his sister — don’t really work.
  • Towards the end — when he gets to the pitch that global warming is a “moral issue” — Gore becomes eloquent and almost moving. Why oh why wasn’t he like this when he ran for president?
  • He had some nice quotes — e.g. “It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so” [Mark Twain] and “You can’t make somebody understand something if their salary depends upon them not understanding it.”[Upton Sinclair]
  • The film contained a clip of George W. Bush discoursing upon Gore: “This guy is so far out in the environmental extreme, we’ll be up to our neck in owls and outta work for every American. He is way out, far out, man.”
  • One fascinating insight. He reported a survey of nearly a thousand peer-reviewed papers on environmental science which looked for evidence of scientific disagreement about the reality and causes of global warming. This turned up precisely zero articles disagreeing with the mainsteam scientific consensus. He then quoted a survey of coverage of global warming in the mainstream media. 53% of the sample either claimed, or conveyed the impression, that there was serious disagreement in the scientific community about the issue. If true, this highlights a serious problem with journalism.
  • My kids were the youngest people in the (Cambridge) audience. They tell me that none of their respective social circles had heard of the film, and they doubted if any one of their friends would go to see it. Sigh.

    I think Larry Lessig was on to something when he started a scheme which enabled people to sponsor other people to go see it.

    My kids also made the point that they are all required to watch Schindler’s List — for very good reasons. They thought that An Inconvenient Truth should be required viewing in UK schools. They’re right.

  • Lottery winners: pay attention

    Behold a classic M-series camera body which takes all those expensive items of glassware that result in Leica owners being unable to feed their families. But instead of film, inside is a 10.3 MP sensor. And where the film-speed indicator used to be, you’ll find a 2.5-inch LCD screen. Available in UK sometime in November.

    Oh — and the price? Er, a mere £2990. Including VAT, naturally.

    “It comes”, says the NYT waspishly, “in black and silver; a rakish fashion photographer’s beret and turtleneck are not included”. Huh! The beret is the only thing some of us can afford.

    Parrot sketch

    Alexander Cockburn, in quasi-sentimental mood..

    I was nearly 30 and yearned for escape. I could see English politics stretching drearily ahead. After Wilson’s return there would be James Callaghan. After Callaghan, Michael Foot. After Foot, Neal Kinnock. After Kinnock…One day in the late summer of 1972 I had occasion to be in the portion of south London known as Balham. It was hot, and the streets infinitely dreary. I must get away, I muttered to myself, like Razumov  talking to Councillor Mikulin  in Conrad’s Under Western Eyes.

    I turned in the direction of the subway station. A dingy sign caught my eye, in a sub-basement window. Parrot readings. I was puzzled. Surely it should be Tarot. I knocked, and the sibyl, in Indian saree, greeted me. She had tarot cards and a parrot, a method of divination with an ancient lineage in India. She dealt the cards. The parrot looked at them, then at me, then at the fortune teller. Some current of energy passed between them. The sybil  paused,  then in a low yet vibrant voice, bodied forth the future to me , disclosing what lay ahead in British public life. Her lips curved around the as yet unfamiliar words “New Labor”. Falteringly, raising her hands before her eyes in trembling dismay at the secret message of the cards, she described a man I know now to have been Tony Blair. I paid her double, then triple as, amid the advisory shrieks of the parrot, she poured out the shape of things to come.

    Within a week, obeying the promptings of the parrot, I had booked a flight to New York and a new life. Ahead of me lay a vast political landscape, seemingly of infinite richness and possibility. Never for a moment have I regretted my journey westward. That parrot in Balham had read the cards correctly. It is probably still alive, and I’m sure that if I were to return for another consultation, it would cry out, “I could have told you so”, and cackle heartily as it described the blasted expectations raised by Democrats stretching from Carter to Clinton…

    Thanks to Godfrey Boyle for spotting it.

    A true shaggy dog story

    From BBC NEWS

    A breakdown patrol man who came to the rescue of a woman motorist has managed to get her car started using her dog.

    Juliette Piesley, 39, had changed the battery in her electronic key fob but was then unable to start her car.

    When AA patrolman Kevin Gorman arrived at the scene in Addlestone, Surrey, he found its immobiliser chip was missing.

    Ms Piesley said her dog George had eaten something, and realising it was the chip, he put the dog in the front seat and started the car with the key.

    Mr Gorman said: “I was glad to get the car started for the member.

    “They will now have to take George [the dog] with them in the car until things take their natural course.

    “It is the first time that I have had to get a dog to help me to start a car.”

    Misunderstandings

    Free Bruce Schneier talk in LA today, 7PM” is the headline on a Boing Boing post today. My first reaction was: “Huh? Bruce Schneier has been arrested?” And then the correct interpretation dawned.

    Reminds me of that old joke about the graffiti response to notices on walls in public spaces saying “Bill Stickers will be Prosecuted”: BILL STICKERS IS INNOCENT!