The perniciousness of ‘personalisation’

Interesting Scientific American article by Brett Frischmann and Devan Desai on how — paradoxically — personalised stimuli can produce homogenous responses:

This personalized-input-to-homogenous-output (“PIHO”) dynamic is quite common in the digital networked environment. What type of homogenous output would digital tech companies like to produce? Often, companies describe their objective as “engagement,” and that sounds quite nice, as if users are participating actively in very important activities. But what they mean is much narrower. Engagement usually refers to a narrow set of practices that generate data and revenues for the company, directly or via its network of side agreements with advertisers, data brokers, app developers, AI trainers, governments and so on.

For example, Facebook offers highly personalized services on a platform optimized to produce and reinforce a set of simple responses — scrolling the feed, clicking an ad, posting content, liking or sharing a post. These actions generate data, ad revenue, and sustained attention. It’s not that people always perform the same action; that degree of homogeneity and social control is neither necessary for Facebook’s interests nor our concerns. Rather, for many people much of the time, patterns of behavior conform to “engagement” scripts engineered by Facebook.

The point about what the companies actually regard as ‘user engagement’ is a useful reminder of how tech companies have become consummately adept at Orwellian doublespeak and euphemism. “In our time”, Orwell wrote in “Politics and the English Language”, “political speech and writing are largely the defence of the indefensible.” Well, in our time, we have strategic euphemisms like “the sharing economy”, “user engagement” and “connecting people”.