Archive for the 'Technology' Category

Why Facebook and Google are buying into drones

[link] Sunday, April 20th, 2014

This morning’s Observer column.

Back in the bad old days of the cold war, one of the most revered branches of the inexact sciences was Kremlinology. In the west, newspapers, thinktanks and governments retained specialists whose job was to scrutinise every scrap of evidence, gossip and rumour emanating from Moscow in the hope that it would provide some inkling of what the Soviet leadership was up to. Until recently, this particular specialism had apparently gone into terminal decline, but events in Ukraine have led to its urgent reinstatement.

The commercial equivalent of Kremlinology is Google- and Facebook-watching. Although superficially more open than the Putin regime, both organisations are pathologically secretive about their long-term aspirations and strategies. So those of us engaged in this strange spectator-sport are driven to reading stock-market analysts’ reports and other ephemera, which is the technological equivalent of consulting the entrails of recently beheaded chickens.

It’s grisly work but someone has to do it, so let us examine what little we know and see if we can make any sense of it…

LATER: Seb Schmoller, struck by my puzzlement about why Facebook had bought Oculus Rift, sent me a link to an interesting blog post by Donald Clark, who has experience of using Oculus kit.

I’ve played around with the Oculus for some time now – played games, roared around several roller-coasters, had my head chopped off by a guillotine, walked around on the floor of the ocean looking up at a whale and shark, floated around the International Space Station using my rocket pack.
Why do I think it matters? It’s possible, just possible, that this device, or one like it, will change the world we know forever. It will certainly revolutionise the world of entertainment. Flat screen TVs have got as big and sharp as they can get. It is clear that most people do want that big, panoramic experience but there’s a limit with 2D. Climb into that screen, which is what the Oculus allows you to do and you can look around, upwards, over your shoulder. You can them move around, do things and things can be done to you. It’s mind blowing.

The problem that Oculus has is getting to market quickly. Kickstarter was fine, for starting. Sony is right on their shoulder with project Morpheus. With this money they can accelerate R&D, have a massive marketing push and keep the price right…

His conclusion:

This is not only a ‘game’ changer, it’s an experience changer. It will change the way we spend our time, expand our experience and acquire skills. I’ve seen the effect it has with children, teenagers, adults and pensioners. It’s an experience, even at low resolution that can change your life, as you know, when you’ve tried it that it’s coming and when it comes it will be all-embracing. Facebook already has the world at its feet with 1.5 billion users, it now has the world on its head.

Translation: maybe the acquisition make more sense than I though.

Making sense of Snowden

[link] Saturday, April 19th, 2014

This is a fantastic example of how to conduct an academic discussion of a really contentious subject. It brings together academics and NSA people to talk calmly about what’s happened and what it means. The participants are Yochai Benkler, Bruce Schneier, and Jonathan Zittrain of the Berkman Center and John DeLong and Anne Neuberger of the National Security Agency. The conversation is expertly moderated by the Berkman Faculty Director Terry Fisher.

It runs for 90 minutes, but is really worth it. So book some time off and watch.

Some thoughts triggered by it, in no particular order…

  1. Tempting thought it might be, I see little point in demonising the agencies (NSA/GCHQ). Most of the people who work in them are conscientious officials engaged on a mission which they believe to be important and necessary. One interesting aspect of the Snowden revelations is that they contain few, if any, horror stories of “bad apples” or corrupt officials abusing their powers. This doesn’t mean that such scandals don’t exist, but my hunch is that this is very different from, say, what went on in MI5 and the CIA during the Kennedy/Nixon/Reagan eras.
  2. The discussion so far has focussed too much on the details or the surveillance programs, and not enough on what the existence of such programs means for society and democracy.
  3. ‘Oversight’ has been interpreted as checking that the agencies strictly adhere to the rules that have been set for them by legislation and executive order. It seems clear already that much of this oversight has been inadequate and flawed. But there has been very little discussion of democratic oversight of the rule-making process itself. It is important, of course, to ensure that rules set by Parliament or Congress are being obeyed at the execution level. But what is equally important – and thus far under-discussed – is whether the rules that have been created by politicians are themselves wise, effective and proportionate. There is little comfort to be derived from government assurances that everything done by NSA/GCHQ is “lawful” if the laws themselves are flawed.
  4. There is an important difference between espionage and bulk surveillance: the former is directed or targeted; the latter is generalised and indiscriminate.
  5. In a way, the agencies were set an impossible task by politicians in the aftermath of 9/11. “Never again” was both the letter and the spirit of the injunction. Societies must never again be vulnerable to the terrible things that terrorists might dream up and conspire about. Charged with this terrible responsibility the agencies attempted to forewarn against any conceivable threat, and the only way they could invent to do that involved the kind of comprehensive surveillance that Snowden reveals. What we don’t know – yet – is whether the agencies were actually doing this kind of surveillance before 9/11, in which case there would be some further awkward questions to be asked.
  6. The “war on terror” proved to be a really pernicious ploy. A state of war implies an existential threat to the nation, which justifies and legitimates very drastic measures. Between 1939 and 1945, for example, Britain was effectively a totalitarian state, and all kinds of civil liberties were drastically curtailed and infringed; but the citizenry grudgingly or willingly accepted these conditions because they understood the existential threat. But the “war on terror” is not a war in that sense; it’s merely a rhetorical device. it did, however provide ideological – and in some cases legal – cover for massive extensions of intrusive surveillance.
  7. Secrecy is always a tricky concept for democracies Because, on the one hand, democracy requires openness and publicity (to ensure that citizens can give their consent to what is being done in their name by state actors); but at the same time, democracies may legitimately need to engage in some activities which have to be kept secret. In some cases, secrecy is legitimate: in 1963, for example, the Cuban missile crisis was resolved by President Kennedy’s decision to offer the prospect of withdrawal of American missiles based in Turkey in return for a Soviet decision to withdraw their missiles from Cuba. This offer was kept secret from the American public for the very good reason that if it had been made public then it might have undermined congressional and public support for the president’s handling of the crisis.
  8. Democracies therefore are always trying to strike a balance between openness and secrecy. This can be a very hard balance to strike, so not surprisingly democracies tend to fudge the issue by offering to lift the veil of secrecy just far enough to provide a semblance of accountability. One of the things we have learned from the Snowdon affair is how threadbare this semblance is. What we have, as one shrewd commentator observed, is not real oversight but “oversight theatre”.
  9. A useful way to conceptualise the problem is to imagine a horizontal line. Activities above the line – for example legislative rule-making – take place in public. This is where policy is formed. Below the line is the area of policy execution by the agencies, and is hidden from the public.
  10. It would be naive to assume that the agencies confine themselves just to execution. They may sometimes be active above the line – for example in framing legislation which meets their needs but which is couched in terms that conceal from an ignorant public and a complacent or incompetent legislature the real import of the legislation. This process has been especially visible since 9/11. In that context, it’s interesting that the legislator who co-authored the Patriot Act has publicly declared his dismay at discovering (pace Snowden) what his statute has supposedly authorised. And in Britain it’s clear that directors of security organisations can play an important role in framing legislation.
  11. In Britain there is a deeply-ingrained tradition of political deference to the security services. This could be because Britain is a society that is more hierarchical and deferential than most. Or it could be that sentiment rules: GCHQ, for example, is seen as the spiritual heir of the wartime Bletchley Park codebreakers, and thus rides on their heroic coat-tails. Whatever the explanation, there are suspicions that budgetary and other proposals from senior security officials receive more favourable treatment in Whitehall than do comparable demands from “civilian” departments. One former senior member of the Blair government told me that in all his time in the Cabinet he could not recall a single instance in which a request from MI5/MI6/GCHQ was turned down by Tony Blair.
  12. Politicians in most Western democracies – including the United States and United Kingdom – are astonishingly ignorant about the capabilities and potential of computing and communication technologies. The proposition that such politicians might be capable of maintaining effective ‘oversight’ of technologically-adept agencies is implausible.
  13. Allied to politicians’ technological ignorance is the fact that “hacker culture” is an entirely alien world to them. This is important in considering the possibility of “mission creep” by surveillance agencies which are staffed by large numbers of talented software engineers. The Snowdon revelations include a few examples of what programmers call “cool hacks” which are indicative of technological exuberance and associated mission creep.
  14. Even if we except that the NSA has strictly adhered to the rules laid down by Congress, there is the problem that some of the activities revealed by Snowdon are nowhere mentioned in the rules. Congress, for example, did not mandate that the RSA encryption which supposedly secures the bulk of the commerce transactions on the open Internet should be covertly compromised by the agency. Nor did Congress mandate that the NSA should approach Microsoft after it acquired Skype with the demand/request that the technology should be modified in order to facilitate surveillance of VoIP communications.
  15. One of the most perplexing aspects of the whole surveillance question is why citizens of some of the most-surveilled societies seem relatively relaxed about it. There are, of course, cultural differences at work here – Germans, for example, seem to much more concerned about the Snowdon revelations than are Britons.
  16. The Snowdon revelations demonstrate the extent to which what one might call the National Surveillance State is a public-private enterprise. In a sense the state has covertly outsourced some of the surveillance to major Internet companies and telecommunications organisations. This is hardly surprising given that the core business of both the NSA/GCHQ and the Internet giants (Google, Yahoo, Facebook, Microsoft) is intensive, detailed, comprehensive surveillance. The only real difference is that the companies claim it is being done with the consent of their users – as registered by their acceptance of the terms and conditions imposed by corporate EULAs (End User Licence Agreements).
  17. One strange aspect of the whole business is the way the US government appeared unaware to the threat that exposure of NSA activities would pose to the country’s big technology companies. It’s inconceivable that policy makers would not have considered the damage that exposure would do. Or is it? Was it just that (see earlier comment about the cluelessness of politicians in this area) that the risk never crossed what might loosely be called their minds?
  18. The biggest question of all — and the one least discussed – is whether the kind of comprehensive surveillance revealed by Snowden and other whistleblowers is compatible with any meaningful conception of democracy.

Triumph of the Nerds

[link] Monday, April 7th, 2014

My Observer piece on Michael Lewis’s new book.

Light travels at 186,000 miles a second in a vacuum, which is another way of saying that it covers 186 miles in a milli-second – a thousandth of a second. Given that much of our contemporary electronic communications are conveyed by pulses of light travelling along fibreoptic cables, we are given to extravagant hyperbole about the “death of distance”. After all, if a message – or a file – can traverse the globe in the blink of an eye, it doesn’t matter whether your hard drive is on your desktop or in a server farm in Nebraska or Sweden.

But it turns out that the speed of light is of great practical interest to some people. One group of them have shelled out $300m to lay a fibreoptic cable in a straight line from Chicago to New York. This involves, among other things, drilling through mountains and under urban areas. And for what? So that the time taken to send a signal between New York and Chicago could be reduced from 17 milliseconds to 13. For that apparently infinitesimal improvement, stock market traders were willing to pay $14m a year, plus a substantial upfront payment, to use the cable.

Therein lies the tale of Michael Lewis’s enthralling new book, Flash Boys, which joins an elite but growing list of volumes that set out to explain how computing is reshaping our world…

Facebook’s “freakishly accurate” face recognition technology

[link] Saturday, April 5th, 2014

Facial recognition software is almost as good at identifying people as humans are, thanks to Facebook. The Facebook AI team published a paper last week highlighting their achievements with DeepFace, the company’s unsettlingly precise facial recognition program.

DeepFace can identify faces at a 97.25 percent accuracy level, just slightly worse than the average human score of 97.53 percent, as Technology Review noted. The DeepFace system reduced facial recognition software errors by 25 percent compared to earlier versions of the software, which is a vast improvement.

‘Freakishly’ is one way of putting it.


Why Snapchat is interesting

[link] Wednesday, April 2nd, 2014

As usual, danah boyd nails it:

Snapchat offers a different proposition. Everyone gets hung up on how the disappearance of images may (or may not) afford a new kind of privacy. Adults fret about how teens might be using this affordance to share inappropriate (read: sexy) pictures, projecting their own bad habits onto youth. But this is isn’t what makes Snapchat utterly intriguing. What makes Snapchat matter has to do with how it treats attention.

When someone sends you an image/video via Snapchat, they choose how long you get to view the image/video. The underlying message is simple: You’ve got 7 seconds. PAY ATTENTION. And when people do choose to open a Snap, they actually stop what they’re doing and look.

In a digital world where everyone’s flicking through headshots, images, and text without processing any of it, Snapchat asks you to stand still and pay attention to the gift that someone in your network just gave you. As a result, I watch teens choose not to open a Snap the moment they get it because they want to wait for the moment when they can appreciate whatever is behind that closed door. And when they do, I watch them tune out everything else and just concentrate on what’s in front of them. Rather than serving as yet-another distraction, Snapchat invites focus.

Furthermore, in an ecosystem where people “favorite” or “like” content that is inherently unlikeable just to acknowledge that they’ve consumed it, Snapchat simply notifies the creator when the receiver opens it up. This is such a subtle but beautiful way of embedding recognition into the system. Sometimes, a direct response is necessary. Sometimes, we need nothing more than a simple nod, a way of signaling acknowledgement. And that’s precisely why the small little “opened” note will bring a smile to someone’s face even if the recipient never said a word.

Snapchat is a reminder that constraints have a social purpose, that there is beauty in simplicity, and that the ephemeral is valuable. There aren’t many services out there that fundamentally question the default logic of social media and, for that, I think that we all need to pay attention to and acknowledge Snapchat’s moves in this ecosystem.

My idea of a perfect blog post. It’s insightful, thought-provoking and beautifully written.

Michael Lewis on Lightspeed

[link] Tuesday, April 1st, 2014

Michael Lewis is, IMHO, one of the best long-form journalists around and his new book is well up to his usual standard. In many ways, it adheres to the classic Lewis formula: find a scandalous set-up of which most people are blissfully unaware; locate some smart guys who have detected the systemic scam and figured out a way to profit from their ingenuity; and then tell their story.

In this case, the story is basically about the speed of light – or, to be more precise, about how the time-difference (in millionths of a second) that it takes an electronic share transaction to traverse one fibre-optic connection rather than another can provide an exceedingly lucrative trading advantage to those who have the kit and the know-how to exploit it.

In the video clip he explains the nub of the idea but, as always, it’s not so much the story as the way Lewis tells it — which is why his book is a must-read for anyone who cares about this stuff.

Writing about it in Quartz, Matt Phillips quotes from another part of the TV interview

“If it wasn’t complicated, it wouldn’t be allowed to happen,” he says. ”The complexity disguises what is happening. If it’s so complicated you can’t understand it, then you can’t question it.”

“This problem”, Phillips says, “goes beyond stock markets: The US financial system is awash in unnecessary complexity. And the reasons are simple: Complexity is profitable and it keeps regulators at bay. ”The jargon of bankers and banking experts is deliberately impenetrable,” wrote economists Anat Admati and Martin Hellwig in their indispensible The Bankers’ New Clothes: What’s Wrong with Banking and What to Do about It. “This impenetrability helps them confuse policy makers and the public.”

There are some echoes of the sub-prime/CDO scandal in Lewis’s new book, in that the people who are supposed to understand how the system worked had little or no idea what was going on under their corporate noses. He recounts how the ‘good’ guys in his tale discovered this when they sought to enlighten leading figures in the financial world about flash trading:

The most sophisticated investors didn’t know what was going on in their own market. Not the big mutual funds, Fidelity and Vanguard. Not the big money-management firms like T. Rowe Price and Capital Group. Not even the most sophisticated hedge funds. The legendary investor David Einhorn, for instance, was shocked; so was Dan Loeb, another prominent hedge-fund manager.

This is an indicator of a really serious underlying problem in our networked world — the stupendous power that superior knowledge, IQ and technical understanding confers on some people. We are completely dependent on systems that are so complex that virtually nobody understands how they work — and how they can be manipulated and gamed by those who do understand them. The obvious rejoinder is “twas ever thus”, but I think that’s too complacent. What’s different now is that the level of technical expertise needed is beyond the reach or capacity of almost everyone. Which means that the elites who do ‘get’ it — and those who employ them — have colossal advantages.

LATER The book has made a BIG impact, to judge from the media coverage, and mostly the reactions have been complimentary. But there were a few contrary opinions. And Andrew Ross Sorkin, writing in the New York Times made some good points.

There is only one problem with Mr. Lewis’s tale: He reserves blame for the wrong villains. He points mostly to the hedge funds and investment banks engaged in high-frequency trading.

But Mr. Lewis seemingly glosses over the real black hats: the big stock exchanges, which are enabling — and profiting handsomely — from the extra-fast access they are providing to certain investors.

While the big Wall Street banks may have invented high-speed trading, it has gained widespread use because it has been encouraged by stock markets like the New York Stock Exchange, Nasdaq and Bats, an electronic exchange that was a pioneer in this area. These exchanges don’t just passively allow certain investors to connect to their systems. They have created systems and pricing tiers specifically for high-speed trading. They are charging higher rates for faster speeds and more data for select clients. The more you pay, the faster you trade.

That is the real problem: The exchanges have a financial incentive to create an uneven playing field.

Footnote: Readers on IoS devices may not be able to see the video clip, for reasons best known to the late Steve Jobs.

How it works

[link] Monday, March 24th, 2014

Lovely animation.

Snooping is a public health issue

[link] Sunday, March 16th, 2014

This morning’s Observer column.

One of the things that baffles me is why more people are not alarmed by what Edward Snowden has been telling us about the scale and intrusiveness of internet surveillance. My hunch is that this is partly because – strangely – people can’t relate the revelations to things they personally understand.

In the past two weeks, two perceptive commentators have been trying to break through this barrier. One is Cory Doctorow, the science-fiction novelist, who had a terrific essay in the Guardian arguing that instead of increasing our security, government agencies such as the NSA, GCHQ and others are actually undermining it. The essay is worth reading in full, but one part of it stood out for me. It’s about the thriving, underworld online market in malicious software. Nowadays, if some hacker discovers a previously unknown vulnerability in widely used software, that discovery can be very valuable – and people will pay large sums for such “zero-day” exploits. But here’s the creepy bit: sometimes, the purchasers are government agencies that buy these pieces of malware to use as weapons against their enemies.

To most people, this will seem pretty abstruse. But with the imaginative skill of a good writer, Doctorow nails it: “If you discovered,” he writes, “that your government was more interested in weaponising typhus than they were in curing it, you would demand that your government treat your water supply with the gravitas and seriousness that it is due.”

Read on

LATER: Right on cue, another great blog post by Bruce Schneier, putting this stuff in an everyday context:

Imagine that you hired a private detective to eavesdrop on a subject. That detective would plant a bug in that subject’s home, office, and car. He would eavesdrop on his computer. He would listen in on that subject’s conversations, both face to face and remotely, and you would get a report on what was said in those conversations. (This is what President Obama repeatedly reassures us isn’t happening with our phone calls. But am I the only one who finds it suspicious that he always uses very specific words? “The NSA is not listening in on your phone calls.” This leaves open the possibility that the NSA is recording, transcribing, and analyzing your phone calls — and very occasionally reading them. This is far more likely to be true, and something a pedantically minded president could claim he wasn’t lying about.)

Now imagine that you asked that same private detective to put a subject under constant surveillance. You would get a different report, one that included things like where he went, what he did, who he spoke to — and for how long — who he wrote to, what he read, and what he purchased. This is all metadata, data we know the NSA is collecting. So when the president says that it’s only metadata, what you should really hear is that we’re all under constant and ubiquitous surveillance.

What’s missing from much of the discussion about the NSA’s activities is what they’re doing with all of this surveillance data. The newspapers focus on what’s being collected, not on how it’s being analyzed — with the singular exception of the Washington Post story on cell phone location collection. By their nature, cell phones are tracking devices. For a network to connect calls, it needs to know which cell the phone is located in. In an urban area, this narrows a phone’s location to a few blocks. GPS data, transmitted across the network by far too many apps, locates a phone even more precisely. Collecting this data in bulk, which is what the NSA does, effectively puts everyone under physical surveillance.

This is new. Police could always tail a suspect, but now they can tail everyone — suspect or not. And once they’re able to do that, they can perform analyses that weren’t otherwise possible. The Washington Post reported two examples. One, you can look for pairs of phones that move toward each other, turn off for an hour or so, and then turn themselves back on while moving away from each other. In other words, you can look for secret meetings. Two, you can locate specific phones of interest and then look for other phones that move geographically in synch with those phones. In other words, you can look for someone physically tailing someone else. I’m sure there are dozens of other clever analyses you can perform with a database like this. We need more researchers thinking about the possibilities. I can assure you that the world’s intelligence agencies are conducting this research.

Schneier is one of the very best commentators on this stuff. Everything he writes about it is worth reading.

25 things you might not know about the Web

[link] Monday, March 10th, 2014

My way of celebrating the Web’s 25th birthday. In the Observer of March 9.

Why the obsession with “coding” misses the point

[link] Monday, February 17th, 2014

My relatively mild column about the Year of Code fiasco has generated a fair amount of comment, and a good many emails, including some from friends who think I was too willing to give the YoC crowd the benefit of the doubt, and citing Andrew Orlowski’s characteristically caustic take on the matter.

Leaving aside the motives of those involved in the ‘initiative’, a bigger concern (for me at any rate) is that the obsession with “coding” has two significant downsides:

  • it misses the point of the new school curriculum (or which more in a minute); and,
  • it risks alienating the audience that the initiative urgently needs to convince — schoolteachers who are not techies and are probably very nervous about what lies ahead for them as they come to terms with the new computing curriculum.

In her disastrous Newsnight interview, Lottie Dexter (and indeed her tormentor, Paxman) both seemed to think that the only motivations for the ‘coding’ initiative are utilitarian and economic: it was, they seemed to think, about kids being able to get jobs, start companies and thereby boost the prospects of UK Plc.

It’s nothing of the kind. This is first and foremost about citizenship. Today’s schoolchildren will inherit a world that is largely controlled by computers and software. The choice that faces them is “Program or Be Programmed”, as Douglas Rushkoff puts it in his book of the same title. If we don’t educate them about this stuff, then they will wind up as passive users of powerful black boxes that are designed and controlled by small elites, most of them located abroad.

Preparation for citizenship in this new world requires an understanding of how software works, how it is created and controlled, and how it can be changed. We don’t want them to grow up as technologically clueless as the parliamentarians who are supposed to oversee GCHQ; or indeed as Paxman, who at one stage fell back on the old trope about not having to understand electricity in order to replace a light bulb. (The obvious riposte — that light bulbs don’t decide whether you get a mortgage, monitor your private communications or count your vote – was obviously beyond poor Dexter.)

The other aspect of this is that, while learning to program is desirable, it’s not the most important part — which is about having a good critical understanding of the technology. And much though I love Raspberry Pi, teachers can achieve a lot of what I would like to achieve without ever touching a piece of hardware — as the wonderful Computer Science Unplugged project in New Zealand demonstrates.