Corporate candour and public sector cant

Posted on by

The UK Information Commissioner has completed her investigation into the deal between Google DeepMind and the Royal Free Hospital Trust which gave the company access to the health records of 1.6m NHS patients. The Commissioner concluded that:

Royal Free NHS Foundation Trust failed to comply with the Data Protection Act when it provided patient details to Google DeepMind.

The Trust provided personal data of around 1.6 million patients as part of a trial to test an alert, diagnosis and detection system for acute kidney injury.

But an ICO investigation found several shortcomings in how the data was handled, including that patients were not adequately informed that their data would be used as part of the test.

The Trust has been asked to commit to changes ensuring it is acting in line with the law by signing an undertaking.

My Cambridge colleague Julia Powles (now at Cornell) and Hal Hodgson of the Economist did a long and thorough investigation of this secret deal (using conventional investigative tools like Freedom of Information requests). This led to the publication of an excellent, peer-reviewed article on “Google DeepMind and healthcare in an age of algorithms”, published in the Springer journal Health and Technology in March. In the period up to and following publication, the authors were subjected to pretty fierce pushback from DeepMind. It was asserted, for example, that their article contained significant factual errors. But requests for information about these supposed ‘errors’ were not granted. As an observer of this corporate behaviour I was struck — and puzzled — by the divergence between DeepMind’s high-minded, holier-than-thou, corporate self-image and its aggressiveness in public controversy. And I wondered if this was a sign that Google iron had entered DeepMind’s soul. (The company was acquired by the search giant in 2014.)

But now all is sweetness and light, apparently. At any rate, DeepMind’s co-founder, Mustafa Suleyman and Dominic King, the Clinical Lead in DeepMind Health, have this morning published a contrite post on the company Blog. “We welcome the ICO’s thoughtful resolution of this case”, they write, “which we hope will guarantee the ongoing safe and legal handling of patient data for Streams [the codename for the collaboration between the company and the NHS Trust]”.

Although today’s findings are about the Royal Free, we need to reflect on our own actions too. In our determination to achieve quick impact when this work started in 2015, we underestimated the complexity of the NHS and of the rules around patient data, as well as the potential fears about a well-known tech company working in health. We were almost exclusively focused on building tools that nurses and doctors wanted, and thought of our work as technology for clinicians rather than something that needed to be accountable to and shaped by patients, the public and the NHS as a whole. We got that wrong, and we need to do better.

This is an intelligent and welcome response. Admitting to mistakes is the surest way to learn. But it’s amazing how few corporations and other organisations do it.

When I first read the draft of Julia’s and Hal’s paper my first thought was that the record of errors they had uncovered was not the product of malign intent, but rather a symptom of what happens when two groups of enthusiasts (consultants in the Royal Free; AI geeks in DeepMind) who were excited by the potential of machine learning in detecting and treating particular diseases. Each group was unduly overawed by the other, and in their determination to get this exciting partnership rolling they ignored (or perhaps were unaware of) the tedious hurdles that one (rightly) has to surmount if one seeks to use patient data for research. And once they had been caught out, defensive corporate instincts took over, preventing an intelligent response to the researchers’ challenge.

Interestingly, there are intimations of this in today’s DeepMind blog post. For example:

“Our initial legal agreement with the Royal Free in 2015 could have been much more detailed about the specific project underway, as well as the rules we had agreed to follow in handling patient information. We and the Royal Free replaced it in 2016 with a far more comprehensive contract … and we’ve signed similarly strong agreements with other NHS Trusts using Streams.”

“We made a mistake in not publicising our work when it first began in 2015, so we’ve proactively announced and published the contracts for our subsequent NHS partnerships.”

“In our initial rush to collaborate with nurses and doctors to create products that addressed clinical need, we didn’t do enough to make patients and the public aware of our work or invite them to challenge and shape our priorities.”

All good stuff. Now let’s see if they deliver on it.

Their NHS partners, however, are much less contrite — even though they are the focus of the Information Commissioner’s report. The Trust’s mealymouthed response says, in part:

“We have co-operated fully with the ICO’s investigation which began in May 2016 and it is helpful to receive some guidance on the issue about how patient information can be processed to test new technology. We also welcome the decision of the Department of Health to publish updated guidance for the wider NHS in the near future.”

This is pure cant. The Trust broke the law. So to say that “we have co-operated fully” and “it is helpful to receive some guidance on the issue about how patient information can be processed” is like a burglar claiming credit for co-operating with the cops and expressing gratitude for their advice on how to break-and-enter legally next time.

Links for today

Posted on by
  1. Jennifer Pahlka, who heads Code for America, was invited to the White House ‘tech summit’. Here’s her low key but interesting report.
  2. In a few years, no investors are going to be looking for AI start-ups. Guess why. (Hint: investors are always looking for the New New Thing.)
  3. Is the Internet Broken?. Good discussion from the Aspen Ideas Festival. Answer: No — the Internet isn’t broken. It’s still doing what it was designed to do, namely to connect computers to other computers. It’s those computers — and what people do with them — that’s the problem. Two guys on the panel — Howard Shrobe from MIT and Milo Medin (ex-NASA, now at Google) — really know their stuff.
  4. To tackle Google’s power, regulators have to go after its ownership of data. Excellent piece by Evgeny Morozov from yesterday’s Observer.
  5. Internet regulation: is it time to rein in the tech giants?. Long and thoughtful piece by Charles Arthur.
  6. How contemporary accounts of the Watergate years read in the Trump era.

What Steve (Jobs) hath wrought

Posted on by

My Observer column on the tenth anniversary of the iPhone:

The iPhone made Apple the world’s most valuable company (with a market capitalisation of $771.44bn when I last checked) but, in a way, that’s the least interesting thing about it. What’s more significant is that it sparked off the smartphone revolution that changed the way people accessed the internet. Steve Jobs’s seminal insight was that a mobile phone could be a powerful, networked handheld device which could also be used to make voice calls. Turning that insight into a marketable reality was a remarkable achievement – commemorated last week by the Computer History Museum in a fascinating two-hour series of recorded conversations with the engineers who built the phone.

The result of this revolution is a world in which most people carry their internet connection around in their bags and pockets. It’s a world of ubiquitous connectivity in which people are never offline and are increasingly addicted to their devices. It’s got to the point where someone has coined a new term – smombies (zombies on smartphones) – to describe pedestrians who walk into obstacles because they are looking at screens rather than at where they’re going…

Read on

Paranoia in the Valley

Posted on by

My Observer piece about US reaction to the Google fine:

The whopping €2.4bn fine levied by the European commission on Google for abusing its dominance as a search engine has taken Silicon Valley aback. It has also reignited American paranoia about the motives of European regulators, whom many Valley types seem to regard as stooges of Mathias Döpfner, the chief executive of German media group Axel Springer, president of the Federation of German Newspaper Publishers and a fierce critic of Google.

US paranoia is expressed in various registers. They range from President Obama’s observation in 2015 that “all the Silicon Valley companies that are doing business there [Europe] find themselves challenged, in some cases not completely sincerely. Because some of those countries have their own companies who want to displace ours”, to the furious off-the-record outbursts from senior tech executives after some EU agency or other has dared to challenge the supremacy of a US-based tech giant.

The overall tenor of these rants (based on personal experience of being on the receiving end) runs as follows. First, you Europeans don’t “get” tech; second, you don’t like or understand innovation; and third, you’re maddened by envy because none of you schmucks has been able to come up with a world-beating tech company…

Read on

So who decides if someone is a journalist or not?

Posted on by

Very interesting snippet in Sue Halpern’s thoughtful NYRB review of Laura Poitras’s film about Julian Assange:

The danger of carving off WikiLeaks from the rest of journalism, as the attorney general may attempt to do, is that ultimately it leaves all publications vulnerable to prosecution. Once an exception is made, a rule will be too, and the rule in this case will be that the government can determine what constitutes real journalism and what does not, and which publications, films, writers, editors, and filmmakers are protected under the First Amendment, and which are not.

This is where censorship begins. No matter what one thinks of Julian Assange personally, or of WikiLeaks’s reckless publication practices, like it or not, they have become the litmus test of our commitment to free speech. If the government successfully prosecutes WikiLeaks for publishing classified information, why not, then, “the failed New York Times,” as the president likes to call it, or any news organization or journalist? It’s a slippery slope leading to a sheer cliff. That is the real risk being presented here, though Poitras doesn’t directly address it.

Tyler Cowen on the iPhone

Posted on by

Like many others, I’ve been reflecting on the tenth anniversary of the iPhone. The Computer History Museum ran a fascinating two-hour show in which John Markoff talked to some of the people who designed and built the first phone. Tyler Cowen also devoted his latest Bloomberg column to it and, as usual, he’s come up with an angle that I hadn’t thought about much:

First, we’ve learned that, even in this age of bits and bytes, materials innovation still matters. The iPhone is behind the scenes a triumph of mining science, with a wide variety of raw materials and about 34 billion kilograms (75 billion pounds) of mined rock as an input to date, as discussed by Brian Merchant in his new and excellent book “The One Device: The Secret History of the iPhone”. A single iPhone has behind it the production of 34 kilos of gold ore, with 20.5 grams (0.72 ounces) of cyanide used to extract the most valuable parts of the gold.

Especially impressive as a material is the smooth touch-screen, and the user’s ability to make things happen by sliding, swiping, zooming and pinching it — the “multitouch” function. That advance relied upon particular materials, as the screen is chemically strengthened, made scrape-resistant and embedded with sensitive sensors. Multitouch wasn’t new, but Apple understood how to build it into a highly useful product.

Imagine putting together a production system that could make 1.2 billion of these devices.

Ten years on

Posted on by

Infographic: 1.2 Billion iPhones in 10 Years | Statista You will find more statistics at Statista

Ten years ago today, the first iPhone went on sale. Is it the most successful product of all time? Probably not. But it’s definitely one of the most influential, in that it was the first real smartphone and thereby kicked off the technology that has shaped the way the Internet — and society — has evolved ever since. And of course it also made Apple the most valuable company in the world.