Moral crumple zones

This morning’s Observer column:

This mindset prompts Dr Elish to coin the term “moral crumple zone” to describe the role assigned to humans who find themselves in the positions that the Three Mile Island operators, the Air France pilots – and the safety driver in the Uber car – occupied. It describes how responsibility for an action may be wrongly attributed to a human being who had limited control over the behaviour of an automated or autonomous system.

“While the crumple zone in a car is meant to protect the human driver,” she writes, “the moral crumple zone protects the integrity of the technological system, at the expense of the nearest human operator. What is unique about the concept of a moral crumple zone is that it highlights how structural features of a system and the media’s portrayal of accidents may inadvertently take advantage of human operators (and their tendency to become “liability sponges”) to fill the gaps in accountability that may arise in the context of new and complex systems.”

Read on

The ‘horseless carriage’ morphs into the horse

From Kara Swisher:

I will die before I buy another car.

I don’t say that because I am particularly old or sick, but because I am at the front end of one of the next major secular trends in tech. Owning a car will soon be like owning a horse — a quaint hobby, an interesting rarity and a cool thing to take out for a spin on the weekend.

Before you object, let me be clear: I will drive in cars until I die. But the concept of actually purchasing, maintaining, insuring and garaging an automobile in the next few decades?


Swisher has form in this area. Many years ago, long before the smartphone, she cancelled her landline phone contract on the grounds that in due course most other people would do so too. (After all, why should phones be tethered to the wall, like goats?) The statistics on how many younger people only have a mobile phone confirm her far-sighted hunch. As far as cars are concerned, though, it’ll probably come down to whether you are an urban or a rural dweller — which partly explains the gilets jaunes crisis in France.

Mainstreaming atrocity

This morning’s Observer column:

The most worrying thought that comes from immersion in accounts of the tech companies’ struggle against the deluge of uploads is not so much that murderous fanatics seek publicity and notoriety from livestreaming their atrocities on the internet, but that astonishing numbers of other people are not just receptive to their messages, but seem determined to boost and amplify their impact by “sharing” them.

And not just sharing them in the sense of pressing the “share” button. What YouTube engineers found was that the deluge contained lots of copies and clips of the Christchurch video that had been deliberately tweaked so that they would not be detected by the company’s AI systems. A simple way of doing this, it turned out, was to upload a video recording of a computer screen taken from an angle. The content comes over loud and clear, but the automated filter doesn’t recognise it.

That there are perhaps tens – perhaps hundreds – of thousands of people across the world who will do this kind of thing is a really scary discovery…

Read on