If you want reproducible science, the software needs to be open source

Posted on by

An increasing proportion of scientific research is data-intensive, and analysing torrents of data requires software, much (if not most) of which is custom-written by researchers to meet their needs. What that means is that computer code has become the equivalent of lab apparatus for some kinds of science. But scientific method requires that the relevant disciplinary community should be able to reproduce an experiment. That means that the custom-written software should also be made available in an accessible form. But often it isn’t — which is why it’s good new to learn of a Nature Editorial arguing that it should. ArsTechnica has a useful piece about this issue. Excerpt:

Modern scientific and engineering research relies heavily on computer programs, which analyze experimental data and run simulations. In fact, you would be hard-pressed to find a scientific paper (outside of pure theory) that didn’t involve code in some way. Unfortunately, most code written for research remains closed, even if the code itself is the subject of a published scientific paper. According to an editorial in Nature, this hinders reproducibility, a fundamental principle of the scientific method.

Reproducibility refers to the ability to repeat some work and obtain similar results. It is especially important when the results are unexpected or appear to defy accepted theories (for example, the recent faster-than-light neutrinos). Scientific papers include detailed descriptions of experimental methods—sometimes down to the specific equipment used—so that others can independently verify results and build upon the work.

Reproducibility becomes more difficult when results rely on software. The authors of the editorial argue that, unless research code is open sourced, reproducing results on different software/hardware configurations is impossible. The lack of access to the code also keeps independent researchers from checking minor portions of programs (such as sets of equations) against their own work.

The Idea Factory

Posted on by

Nobody who writes about the history of computing can ignore Bell Labs, that astonishing institution in New Jersey that created so much of the technology we nowadays take for granted. An interesting essay in the NYT has brought it back into focus for me because I’m fascinated by the problem of how to manage creative people in such a way that their creativity is liberated, not stifled, by the organisation that funds them. (Many years ago I co-authored a paper on the subject with Bob Taylor — the guy who funded the ARPAnet and later ran the Computer Systems Lab at Xerox PARC during the time when its researchers invented most of the computing technology we use today. The title of our essay was “Zen and the Art of Research Management” and it was published in December 2003 in a volume of essays dedicated to Roger Needham.)

The NYT article is by Jon Gertner, who is the author of a forthcoming book on Bell Labs entitled The Idea Factory: Bell Labs and the Great Age of American Innovation. It’s on my wish list.

At Bell Labs, the man most responsible for the culture of creativity was Mervin Kelly. Probably Mr. Kelly’s name does not ring a bell. Born in rural Missouri to a working-class family and then educated as a physicist at the University of Chicago, he went on to join the research corps at AT&T. Between 1925 and 1959, Mr. Kelly was employed at Bell Labs, rising from researcher to chairman of the board. In 1950, he traveled around Europe, delivering a presentation that explained to audiences how his laboratory worked.

His fundamental belief was that an “institute of creative technology” like his own needed a “critical mass” of talented people to foster a busy exchange of ideas. But innovation required much more than that. Mr. Kelly was convinced that physical proximity was everything; phone calls alone wouldn’t do. Quite intentionally, Bell Labs housed thinkers and doers under one roof. Purposefully mixed together on the transistor project were physicists, metallurgists and electrical engineers; side by side were specialists in theory, experimentation and manufacturing. Like an able concert hall conductor, he sought a harmony, and sometimes a tension, between scientific disciplines; between researchers and developers; and between soloists and groups.

ONE element of his approach was architectural. He personally helped design a building in Murray Hill, N.J., opened in 1941, where everyone would interact with one another. Some of the hallways in the building were designed to be so long that to look down their length was to see the end disappear at a vanishing point. Traveling the hall’s length without encountering a number of acquaintances, problems, diversions and ideas was almost impossible. A physicist on his way to lunch in the cafeteria was like a magnet rolling past iron filings…

Cracking the penal code

Posted on by

Fascinating story in the Economist about the case of Sergey Aleynikov, a Goldman Sachs programmer, who was convicted in December 2010 of stealing code tied to Goldman’s lucrative high-speed proprietary-trading operations for use by a new employer.

On February 16th, after he had spent nearly a year in prison, three judges in a federal appeals court unanimously reversed his conviction in a hearing that lasted just a single morning. Their written opinion is now eagerly awaited.

Mr Aleynikov admitted to taking code with him on his way out of Goldman, but argued successfully that this did not constitute a crime, or, to be more specific, a federal crime. He benefited from the help of a thorough lawyer, who adroitly knocked down two key claims. Because the computer trading system was not licensed or offered for sale, claimed Kevin Marino, the defendant’s lawyer, it was not a product to be bought or sold for interstate commerce, a key provision for a federal case. Because computer coding constitutes intangible intellectual property, Mr Marino said, it did not qualify under the goods, wares or merchandise components that are protected under the corporate-espionage act.

The judges quickly accepted these arguments. It is possible that lesser charges could be brought in a state court by a different prosecutor. But as it stands, the ruling raises questions about what sort of legal protection financial firms enjoy for technical knowledge that has become as important as capital or clients and that sits with a few highly mobile employees. The banks may have no choice but to inspire loyalty in their programmers so they don’t leave in the first place.

Changing one’s mind

Posted on by

I’ve just finished Adam Sisman’s magnificent biography of Hugh Trevor-Roper. It seems to me to be a model of its kind – knowledgeable, properly referenced, sympathetic without being sycophantic and extremely readable. And it caused me to revise my opinion of its subject. I’ve always been interested by Trevor-Roper, partly because I’ve been fascinated by history since I was a kid (it was a toss-up whether I would read history or engineering when I first went to university) and by historians (some of my best friends are practitioners of that craft). But I was particularly intrigued by T-R because of his literary style — which was waspish and mischievous to a degree rare even among historians. (Reading their observations of one another’s failings always reminds me of Evelyn Waugh’s observation that “the only virtue schoolboys demand from one another is humility”.)

It’s not surprising that Waugh came to mind. For one thing, he hated T-R because of his hostility to Catholicism. But Waugh also had the same kind of wickedly sarcastic prose style. Witness his observation when an acquaintance announced in White’s one day that surgeons had operated on Randolph Churchill (an authentic Grade One monster if ever there was one) to remove a tumour that had then been found to be benign. “Ah”, said Waugh, “the wonders of medical science: to have found the only part of Randolph that was not malignant — and to have removed it!.”

T-R shared another characteristic with Waugh in that both combined acute snobbery with assiduous social climbing: they both came from middle-class backgrounds and envied folks with what they regarded as better pedigrees. (When someone once tried to cheer Waugh up by reminding him that one of his ancestors had been ennobled in an act of political patronage, he responded that he would like to have been descended from a “useless peer”.)

Since Waugh was a deeply unpleasant person, I had always assumed that Roper must also have been similarly obnoxious. That impression was reinforced by reading his collected letters to Bernard Berenson, which are often hilarious, but also riddled with revolting sycophancy towards the old fraud.

Adam Sisman’s book has made me revise that impression. He reveals that T-R seems to have had a real gift for friendship, that he could be very loyal, principled and courageous at times and that he was generous with his time and support for his students and protégés. Sisman also illuminates one of the great mysteries of T-R’s life, which is how someone who was so gifted never managed to produce the magnum opus that most people expected of him. The answer seems to be that he allowed himself constantly to be diverted by interesting journalistic projects, most of which stemmed from the fact that, early in his career, he had written a masterly, definitive account of the last days of Hitler. As an academic who has often been similarly distracted by journalism, I rather empathised with that predicament!

The Last Days of Hitler is a wonderful piece of factual journalism, which I would recommend to anyone aspiring to practice that grisly trade. Since most of us never produce anything as good as that, T-R can be forgiven a lot. But in a way the most illuminating part of Sisman’s book comes towards the end, when he deals with T-R’s time as Master of Peterhouse and his catastrophic error of judgement in authenticating the “Hitler diaries”. I know something of his time at Peterhouse, but hadn’t fully realised how courageous and effective he was in dealing with what was, at that time, the nastiest nest of donnish vipers in Oxbridge. No doubt some of Sisman’s chronicle had to be toned down because of the libel laws, so we can look forward to an unexpurgated second edition when the last of the aforementioned vipers has passed to his reward in Hell.

I was unexpectedly moved by the Hitler Diaries fiasco as recounted by Sisman. It was, of course, a catastrophe for T-R, and a mighty boon for all those who had, over the years, had to endure the sharp end of his wit. But what is impressive in retrospect is the courageous way he shouldered the blame and the ridicule — in sharp contrast to Murdoch and his minions who mostly tried to dodge their responsibility for the error. It’s not often that a book makes one change one’s mind. But this one did.

Ole King Coal

Posted on by

Gloomy news from the Economist.

Feb 25th 2012 | from the print edition

“OUR civilisation”, wrote George Orwell over 70 years ago, “is founded on coal.” Unlike Europe’s, Asia’s still is. In 2010, according to the International Energy Agency (IEA), a think-tank, coal accounted for just one-fifth of primary energy supply in the OECD countries. But, in the world as a whole, coal accounted for almost half of the increase in energy use from 2000-10. Coal, says Edward Cunningham of Boston University, is experiencing an “historically incredible” resurgence, and may even overtake oil as a fuel by 2025. There is plenty of it and, compared with rival fuels, it is cheap. And often dirty.

Asia has been responsible for over two-thirds of the growth in global energy demand over the past two decades. As, above all, China and India race towards prosperity, they will burn coal in huge volumes. The resulting emissions of carbon dioxide will be among the biggest hurdles in the way of a global agreement on limiting climate change…

Present at the Creation

Posted on by

George Dyson has written a fascinating book about the building of the first stored-program computer by John von Neumann and his colleagues at the Institute of Advanced Studies at Princeton. After I’d finished the book I had an email exchange with him, an edited version of which appears in this morning’s Observer.

Once upon a time, a “computer” was a human being, usually female, who did calculations set for her by men in suits. Then, in the 1940s, something happened: computers became machines based on electronics. The switch had awesome implications; in the end, it spawned a technology that became inextricably woven into the fabric of late-20th- and early 21st-century life and is now indispensable. If the billions of (mostly unseen) computers that now run our industrialised support systems were suddenly to stop working, then our societies would very rapidly grind to a halt.

So the question of where this Promethean force sprang from is an intriguing one, as interesting in its way as the origins of the industrial revolution…

Photograph shows the book on sale in Heffers bookshop in Cambridge yesterday.

The seminar

Posted on by


The seminar, originally uploaded by jjn1.

David Howarth, the former MP for Cambridge (and one of the few Members of Parliament to emerge with honour from the expenses scandal), at a CSaP seminar on Friday. He’s now resumed his academic career after standing down at the last election. His successor as MP, Julian Huppert, was an academic until he was elected.

Sovereigns of Cyberspace?

Posted on by

This morning’s Observer column.

One of the central ideas in MacKinnon’s book is the concept of what she calls “sovereigns of cyberspace”, – companies like Facebook, Google, Apple and Amazon that now exercise the kinds of power that were hitherto reserved for real “sovereigns” – governments operating within national jurisdictions. Witness, for example, the way in which Amazon arbitrarily removed Wikileaks from its cloud computing servers without any justification that would have withstood a First Amendment legal challenge ; or the way that Facebook took down a page used by Egyptian activists to co-ordinate protests on the grounds that they had violated the company’s rules by not using their real names.

The powers to curtail people’s freedom of speech in this way were traditionally reserved for governments which – in democracies at least – theoretically derived their legitimacy from John Locke’s notion of “the consent of the governed”. (It’s worth saying that some political scientists balk at the notion of companies as “sovereigns”. After all, Zuckerberg can’t lock you up, whereas a real government could.) The question MacKinnon raises is: in what sense do Jeff Bezos of Amazon, Mark Zuckerberg of Facebook, Larry Page and Sergey Brin of Google enjoy the consent of the networked?

Consent of the Networked

Posted on by

I’m reading Rebecca Mackinnon’s excellent new book — Consent of the Networked: The Worldwide Struggle for Internet Freedom. It’s a sobering, readable, thought-provoking work which, I’d say, will find its way onto a lot of reading lists in the next year or two. She’s had an interesting career — starting as a mainstream (CNN) journalist specialising in China, and moving later to become a scholar of cyberspace. Her work on China’s special brand of “networked authoritarianism” is the best thing we have on that phenomenon. For those who are too busy to tackle the book, this lecture and the Q&A that followed it provide a good introduction to her views. And there’s a good critical review of the book by Adam Thierer here. Rebecca Rosen also has an excellent interview with Mackinnon in The Atlantic.

Are people getting better at managing their FB privacy settings?

Posted on by

Interesting report from Pew Research Center.

A survey by the Pew Research Center’s Internet Project & American Life provides new data about the privacy settings people choose for their social networking profiles, and the specific steps users take to control the flow of information to different people within their networks.

About two-thirds (63%) of adults say they currently maintain a profile on a social networking site. Nearly six-in-ten (58%), say their main profile is set to be private so that only friends can see it; another 19% set their profiles to partially private so that friends of friends or networks can view them; 20% say their main profile is completely public.

The number of social network users who prune and manage their accounts has increased: 63% of them have deleted people from their “friends” lists, up from 56% in 2009; 44% have deleted comments made by others on their profile; and 37% have removed their names from photos that were tagged to identify them.