Sleepwalking into dystopia

This morning’s Observer column:

When the history of our time comes to be written, one of the things that will puzzle historians (assuming any have survived the climate cataclysm) is why we allowed ourselves to sleepwalk into dystopia. Ever since 9/11, it’s been clear that western democracies had embarked on a programme of comprehensive monitoring of their citizenry, usually with erratic and inadequate democratic oversight. But we only began to get a fuller picture of the extent of this surveillance when Edward Snowden broke cover in the summer of 2013.

For a time, the dramatic nature of the Snowden revelations focused public attention on the surveillance activities of the state. In consequence, we stopped thinking about what was going on in the private sector. The various scandals of 2016, and the role that network technology played in the political upheavals of that year, constituted a faint alarm call about what was happening, but in general our peaceful slumbers resumed: we went back to our smartphones and the tech giants continued their appropriation, exploitation and abuse of our personal data without hindrance. And this continued even though a host of academic studies and a powerful book by Shoshana Zuboff showed that, as the cybersecurity guru Bruce Schneier put it, “the business model of the internet is surveillance”.

The mystery is why so many of us are still apparently relaxed about what’s going on…

Read on

The dark underbelly of social media

My Observer review of Behind the Screen, Sarah T. Roberts’s remarkable exploration of the exploitative world of content ‘moderation’.

The best metaphor for the net is to think of it as a mirror held up to human nature. All human life really is there. There’s no ideology, fetish, behaviour, obsession, perversion, eccentricity or fad that doesn’t find expression somewhere online. And while much of what we see reflected back to us is uplifting, banal, intriguing, harmless or fascinating, some of it is truly awful, for the simple reason that human nature is not only infinitely diverse but also sometimes unspeakably cruel.

In the early days of the internet and, later, the web, this didn’t matter so much. But once cyberspace was captured by a few giant platforms, particularly Google, YouTube, Twitter and Facebook, then it became problematic. The business models of these platforms depended on encouraging people to upload content to them in digital torrents. “Broadcast yourself”, remember, was once the motto of YouTube.

And people did – as they slit the throats of hostages in the deserts of Arabia, raped three-year-old girls, shot an old man in the street, firebombed the villages of ethnic minorities or hanged themselves on camera…

All of which posed a problem for the social media brands, which liked to present themselves as facilitators of creativity, connectivity and good clean fun, an image threatened by the tide of crud that was coming at them. So they started employing people to filter and manage it. They were called “moderators” and for a long time they were kept firmly under wraps, so that nobody knew about them.

That cloak of invisibility began to fray as journalists and scholars started to probe this dark underbelly of social media…

Read on

Fines don’t work. To control tech companies we have to hit them where it really hurts

Today’s Observer comment piece

If you want a measure of the problem society will have in controlling the tech giants, then ponder this: as it has become clear that the US Federal Trade Commission is about to impose a fine of $5bn (£4bn) on Facebook for violating a decree governing privacy breaches, the company’s share price went up!

This is a landmark moment. It’s the biggest ever fine imposed by the FTC, the body set up to police American capitalism. And $5bn is a lot of money in anybody’s language. Anybody’s but Facebook’s. It represents just a month of revenues and the stock market knew it. Facebook’s capitalisation went up $6bn with the news. This was a fine that actually increased Mark Zuckerberg’s personal wealth…

Read on

How Silicon Valley lost its shine

This morning’s Observer column:

Remember the time when tech companies were cool? So do I. Once upon a time, Silicon Valley was the jewel in the American crown, a magnet for high IQ – and predominately male – talent from all over the world. Palo Alto was the centre of what its more delusional inhabitants regarded as the Florence of Renaissance 2.0. Parents swelled with pride when their offspring landed a job with the Googles, Facebooks and Apples of that world, where they stood a sporting chance of becoming as rich as they might have done if they had joined Goldman Sachs or Lehman Brothers, but without the moral odium attendant on investment backing. I mean to say, where else could you be employed by a company to which every president, prime minister and aspirant politician craved an invitation? Where else could you be part of inventing the future?

But that was then and this is now…

Read on

Getting things into perspective

From Zeynep Tufecki:

We don’t have to be resigned to the status quo. Facebook is only 13 years old, Twitter 11, and even Google is but 19. At this moment in the evolution of the auto industry, there were still no seat belts, airbags, emission controls, or mandatory crumple zones. The rules and incentive structures underlying how attention and surveillance work on the internet need to change. But in fairness to Facebook and Google and Twitter, while there’s a lot they could do better, the public outcry demanding that they fix all these problems is fundamentally mistaken. There are few solutions to the problems of digital discourse that don’t involve huge trade-offs—and those are not choices for Mark Zuckerberg alone to make. These are deeply political decisions. In the 20th century, the US passed laws that outlawed lead in paint and gasoline, that defined how much privacy a landlord needs to give his tenants, and that determined how much a phone company can surveil its customers. We can decide how we want to handle digital surveillance, attention-channeling, harassment, data collection, and algorithmic decision­making. We just need to start the discussion. Now.

Focussing on the difficulty of ‘moderating’ vile content obscures the real problem

Good OpEd piece by Charlie Warzel:

Focusing only on moderation means that Facebook, YouTube and other platforms, such as Reddit, don’t have to answer for the ways in which their platforms are meticulously engineered to encourage the creation of incendiary content, rewarding it with eyeballs, likes and, in some cases, ad dollars. Or how that reward system creates a feedback loop that slowly pushes unsuspecting users further down a rabbit hole toward extremist ideas and communities.

On Facebook or Reddit this might mean the ways in which people are encouraged to share propaganda, divisive misinformation or violent images in order to amass likes and shares. It might mean the creation of private communities in which toxic ideologies are allowed to foment, unchecked. On YouTube, the same incentives have created cottage industries of shock jocks and livestreaming communities dedicated to bigotry cloaked in amateur philosophy.

The YouTube personalities and the communities that spring up around the videos become important recruiting tools for the far-right fringes. In some cases, new features like “Super Chat,” which allows viewers to donate to YouTube personalities during livestreams, have become major fund-raising tools for the platform’s worst users — essentially acting as online telethons for white nationalists.

Facebook’s targeting engine: still running smoothly on all cylinders

Well, well. Months — years — after the various experiments with Facebook’s targeting engine showing hos good it was at recommending unsavoury audiences, this latest report by the Los Angeles Times shows that it’s lost none of its imaginative acuity.

Despite promises of greater oversight following past advertising scandals, a Times review shows that Facebook has continued to allow advertisers to target hundreds of thousands of users the social media firm believes are curious about topics such as “Joseph Goebbels,” “Josef Mengele,” “Heinrich Himmler,” the neo-nazi punk band Skrewdriver and Benito Mussolini’s long-defunct National Fascist Party.

Experts say that this practice runs counter to the company’s stated principles and can help fuel radicalization online.

“What you’re describing, where a clear hateful idea or narrative can be amplified to reach more people, is exactly what they said they don’t want to do and what they need to be held accountable for,” said Oren Segal, director of the Anti-Defamation League’s center on extremism.

Note also, that the formulaic Facebook response hasn’t changed either:

After being contacted by The Times, Facebook said that it would remove many of the audience groupings from its ad platform.

“Most of these targeting options are against our policies and should have been caught and removed sooner,” said Facebook spokesman Joe Osborne. “While we have an ongoing review of our targeting options, we clearly need to do more, so we’re taking a broader look at our policies and detection methods.”

Ah, yes. That ‘broader look’ again.

Facebook: the regulatory noose tightens

This is a big day. The DCMS Select Committee has published its scarifying report into Facebook’s sociopathic exploitation of its users’ data and its cavalier attitude towards both legislators and the law. As I write, it is reportedly negotiating with the Federal Trade Commission (FTC) — the US regulator — on the multi-billion-dollar fine the agency is likely to levy on the company for breaking its 2011 Consent Decree.

Couldn’t happen to nastier people.

In the meantime, for those who don’t have the time to read the 110-page DCMS report, Techcrunch has a rather impressive and helpful summary — provided you don’t mind the rather oppressive GDPR spiel that accompanies it.

The perniciousness of ‘personalisation’

Interesting Scientific American article by Brett Frischmann and Devan Desai on how — paradoxically — personalised stimuli can produce homogenous responses:

This personalized-input-to-homogenous-output (“PIHO”) dynamic is quite common in the digital networked environment. What type of homogenous output would digital tech companies like to produce? Often, companies describe their objective as “engagement,” and that sounds quite nice, as if users are participating actively in very important activities. But what they mean is much narrower. Engagement usually refers to a narrow set of practices that generate data and revenues for the company, directly or via its network of side agreements with advertisers, data brokers, app developers, AI trainers, governments and so on.

For example, Facebook offers highly personalized services on a platform optimized to produce and reinforce a set of simple responses — scrolling the feed, clicking an ad, posting content, liking or sharing a post. These actions generate data, ad revenue, and sustained attention. It’s not that people always perform the same action; that degree of homogeneity and social control is neither necessary for Facebook’s interests nor our concerns. Rather, for many people much of the time, patterns of behavior conform to “engagement” scripts engineered by Facebook.

The point about what the companies actually regard as ‘user engagement’ is a useful reminder of how tech companies have become consummately adept at Orwellian doublespeak and euphemism. “In our time”, Orwell wrote in “Politics and the English Language”, “political speech and writing are largely the defence of the indefensible.” Well, in our time, we have strategic euphemisms like “the sharing economy”, “user engagement” and “connecting people”.