I’m collecting these for a talk I’m giving on Big Data’s obsession with correlation rather than causation.
Category Archives: Big Data
Our Kafkaesque world
This morning’s Observer column.
When searching for an adjective to describe our comprehensively surveilled networked world – the one bookmarked by the NSA at one end and by Google, Facebook, Yahoo and co at the other – “Orwellian” is the word that people generally reach for.
But “Kafkaesque” seems more appropriate. The term is conventionally defined as “having a nightmarishly complex, bizarre, or illogical quality”, but Frederick Karl, Franz Kafka’s most assiduous biographer, regarded that as missing the point. “What’s Kafkaesque,” he once told the New York Times, “is when you enter a surreal world in which all your control patterns, all your plans, the whole way in which you have configured your own behaviour, begins to fall to pieces, when you find yourself against a force that does not lend itself to the way you perceive the world.”
A vivid description of this was provided recently by Janet Vertesi, a sociologist at Princeton University. She gave a talk at a conference describing her experience of trying to keep her pregnancy secret from marketers…
Big Data and the Hype Cycle
This morning’s Observer column.
As the “big data” bandwagon gathers steam, it’s appropriate to ask where it currently sits on the hype cycle. The answer depends on which domain of application we’re talking about. If it’s the application of large-scale data analytics for commercial purposes, then many of the big corporations, especially the internet giants, are already into phase four. The same holds if the domain consists of the data-intensive sciences such as genomics, astrophysics and particle physics: the torrents of data being generated in these fields lie far beyond the processing capabilities of mere humans.
But the big data evangelists have wider horizons than science and business: they see the technology as a tool for increasing our understanding of society and human behaviour and for improving public policy-making. After all, if your shtick is “evidence-based policy-making”, then the more evidence you have, the better. And since big data can provide tons of evidence, what’s not to like?
So where on the hype cycle do societal applications of big data technology currently sit? The answer is phase one, the rapid ascent to the peak of inflated expectations, that period when people believe every positive rumour they hear and are deaf to sceptics and critics…