What’s the point of The Social Network?

Posted on by

This morning’s Observer column.

Lessig’s point is that it’s the open internet that should be the real hero of the story. “What’s important here,” he writes, “is that Zuckerberg’s genius could be embraced by half-a-billion people within six years of its first being launched, without – and here is the critical bit – asking the permission of anyone.” That’s true, but I think Lessig is too harsh. The message he wants the film to communicate is there in the screenplay if you look hard enough. It lies in the film’s portrayal of the contrast between what happens to unauthorised innovation on a closed, tightly controlled system and what’s possible with the open, uncontrolled architecture of the internet.

A systems view of digital preservation

Posted on by

The longer I’ve been around, the more concerned I become about long-term data loss — in the archival sense. What are the chances that the digital record of our current period will still be accessible in 300 years’ time? The honest answer is that we don’t know. And my guess is that it definitely won’t be available unless we take pretty rigorous steps to ensure it. Otherwise it’s posterity be damned.

It’s a big mistake to think about this as a technical problem — to regard it as a matter of bit-rot, digital media and formats. If anything, the technical aspects are the trivial aspects of the problem. The really hard questions are institutional: how can we ensure that there are organisations in place in 300 years that will be capable of taking responsibility for keeping the archive intact, safe and accessible?

Aaron Schwartz has written a really thoughtful blog post about this in which he addresses both the technical and institutional aspects. About the latter, he has this to say:

Recall that we have at least three sites in three political jurisdictions. Each site should be operated by an independent organization in that political jurisdiction. Each board should be governed by respected community members with an interest in preservation. Each board should have at least five seats and move quickly to fill any vacancies. An engineer would supervise the systems, an executive director would supervise the engineer, the board would supervise the executive director, and the public would supervise the board.

There are some basic fixed costs for operating such a system. One should calculate the high-end estimate for such costs along with high-end estimates of their growth rate and low-end estimates of the riskless interest rate and set up an endowment in that amount. The endowment would be distributed evenly to each board who would invest it in riskless securities (probably in banks whose deposits are ensured by their political systems).

Whenever someone wants to add something to the collection, you use the same procedure to figure out what to charge them, calculating the high-end cost of maintaining that much more data, and add that fee to the endowments (split evenly as before).

What would the rough cost of such a system be? Perhaps the board and other basic administrative functions would cost $100,000 a year, and the same for an executive director and an engineer. That would be $300,000 a year. Assuming a riskless real interest rate of 1%, a perpetuity for that amount would cost $30 million. Thus the cost for three such institutions would be around $100 million. Expensive, but not unmanageable. (For comparison, the Internet Archive has an annual budget of $10-15M, so this whole project could be funded until the end of time for about what 6-10 years of the Archive costs.)

Storage costs are trickier because the cost of storage and so on falls so rapidly, but a very conservative estimate would be around $2000 a gigabyte. Again, expensive but not unmanageable. For the price of a laptop, you could have a gigabyte of data preserved for perpetuity.

These are both very high-end estimates. I imagine that were someone to try operating such a system it would quickly become apparent that it could be done for much less. Indeed, I suspect a Mad Archivist could set up such a system using only hobbyist levels of money. You can recruit board members in your free time, setting up the paperwork would be a little annoying but not too expensive, and to get started you’d just need three servers. (I’ll volunteer to write the Python code.) You could then build up the endowment through the interest money left over after your lower-than-expected annual costs. (If annual interest payments ever got truly excessive, the money could go to reducing the accession costs for new material.)

Any Mad Archivists around?

Worth reading in full.

LATER: Dan Gillmor has been attending a symposium at the Library of Congress about preserving user-generated content, and has written a thoughtful piece on Salon.com about it.

The reason for libraries and archives like the Library of Congress is simple: We need a record of who we are and what we’ve said in the public sphere. We build on what we’ve learned; without understanding the past we can’t help but screw up our future.

It was easier for these archiving institutions when media consisted of a relatively small number of publications and, more recently, broadcasts. They’ve always had to make choices, but the volume of digital material is now so enormous, and expanding at a staggering rate, that it won’t be feasible, if it ever really was, for institutions like this to find, much less, collect all the relevant data.

Meanwhile, those of us creating our own media are wondering what will happen to it. We already know we can’t fully rely on technology companies to preserve our data when we create it on their sites. Just keeping backups of what we create can be difficult enough. Ensuring that it’ll remain in the public sphere — assuming we want it to remain there — is practically impossible.

Dan links to another thoughtful piece, this time by Dave Winer. Like Aaron Schwartz, Dave is concerned not just with the technological aspects of the problem, but also with the institutional side. Here are his bullet-points:

1. I want my content to be just like most of the rest of the content on the net. That way any tools create to preserve other people’s stuff will apply to mine.

2. We need long-lived organizations to take part in a system we create to allow people to future-safe their content. Examples include major universities, the US government, insurance companies. The last place we should turn is the tech industry, where entities are decidedly not long-lived. This is probably not a domain for entrepreneurship.

3. If you can afford to pay to future-safe your content, you should. An endowment is the result, which generates annuities, that keeps the archive running.

4. Rather than converting content, it would be better if it was initially created in future-safe form. That way the professor’s archive would already be preserved, from the moment he or she presses Save.

5. The format must be factored for simplicity. Our descendents are going to have to understand it. Let’s not embarass ourselves, or cause them to give up.

6. The format should probably be static HTML.

7. ??

The Millbank ‘riot’

Posted on by

The thought that came to mind watching coverage of the break-in into Tory HQ was whether the more aggressive ‘protestors’ might not, in fact, have been provocateurs, so convenient was the excuse they provided for synthetic right-wing outrage. I spoke to several people who had been on the demonstration in London, and without exception they described it as a heartening, good-natured, serious mobilisation of public opinion which was clearly about more than just the narrow issue of student fees. But you’d never gather that from a survey of the mainstream media.

This point is nicely articulated by Guy Aitchison in his piece on openDemocracy.net:

Now, clearly there were acts of vandalism that accompanied the occupation of Millbank, but the instinct of the crowd was decisively against violence. When one protester picked up a rock, he was told to stop being an “idiot” and the throwing of the fire extinguisher was greeted by a chorus of booes and a chant of “stop throwing shit”, as this video shows. A 23 year old man from Cambridgeshire has apparently now been arrested over this incident. Good. If it is indeed him who hurled the projectile, he had no support for his actions amongst students at the time, and will have none now.

The occupation of 30 Millbank, on the other hand, certainly did have the support of the crowd. This wasn’t just a minority of hotheads, a rogue gang of “anarchists” and “Trots”, as Caroline Flint put it on Question Time yesterday. These were young, fresh-faced kids of the kind you’d find in any student bar. Disillusioned and enraged by a political elite that has chosen to make their generation pay for a crisis they didn’t cause, they saw an opportunity passing Millbank to get involved in a spontaneous direct action against the poorly guarded Tory HQ. And they took it. The hundreds who occupied the building had the support of the thousands who cheered them on outside, and many more no doubt on TV. Many I spoke to, who got involved in the occupation, were 16 and 17 and had taken the day off school, risking the wrath of their teachers, to protest. As John Harris put it:

“What happened on Wednesday afternoon was not some meaningless rent-a-mob flare-up, nor an easily-ignored howl of indignation from some of society’s more privileged citizens. It was an early sign of people growing anxious and restless, and what a government pledged to such drastic plans should increasingly expect.”

The other important point to recognise is that this wasn’t a purely self-interested protest about fees by a privileged few. The majority of those protesting won’t be affected by the hike in fees, and in any case students were keen to show solidarity with other victims of the coalition’s austerity agenda. The slogans and statement by those on the roof of Millbank make this clear. As Richard Seymour points out, it is patronising and untrue, to imply, as Polly Toynbee does, that only the middle class care about defending university education – many students come from working class families, live in poor quality accommodation and struggle to get by on low paid jobs. The benefit of accessible higher education to the individual and society is recognised across all social classes.

The strange thing about the current period of phoney war is how strangely passive enraged populations are, both here and in Ireland (where the public sector cuts necessitated by the recklessness of bankers and their corrupt political associates are much more savage than anything contemplated by the Con-Dem coalition). But the rage is real, and deep-seated. And it will find an outlet, one way or another. In the US, the rage has taken a particular route — in the Tea Party and the mid-term elections. It hasn’t yet been expressed in Europe, except in France and Greece. All it needs is for the appearance of a skilful, charismatic politician on the scene who is willing to exploit and channel the rage and then we’ll see some interesting developments.

FIFA, corruption and the Olympic five-ring circus

Posted on by

It’s not often that I agree with Simon Jenkins, but the news that Dave Cameron is going to FIFA to plead Britain’s case to host the 2018 World Cup has annoyed him almost as much as it annoys me.

Britain should have no truck with a body like Fifa, any more than it should with the International Olympic Committee or those who run much of international sport. Five minutes spent with the cuttings, or trawling such websites as playthegame and transparencyinsport, should have stopped Cameron being photographed shaking hands with Blatter at Downing Street. His staff should have read Andrew Jennings’ Foul! on Blatter, and thrown in Christopher Shaw’s Five Ring Circus, about the IOC, for good measure.

Incident after incident, case after case, has shown these self-governing supranational apparatus riddled with accusations of backhanders, bribes and fixed votes – often quite legal in the countries where they carefully base themselves, such as Switzerland and Liechtenstein. Their officials jet the world, pushing the one narcotic to which all modern governments are addicted – sporting glory.

For decades the IOC turned a blind eye to the communist bloc’s use of drugs to boost performance, or China’s (and Tony Blair’s) exploitation of the Olympics for political ends. It now demands inexcusable outlays of taxpayers’ money to stage its two-week festival of minority sports. When the IOC demands an exclusive “Zil lane” for its official cars up London’s Mile End Road, the British government meekly obeys. It would not offer this to a head of state.

Likewise Fifa turns a blind eye to longstanding charges of vote rigging. Its laughable ethics committee deplored last month’s revelations as “unethical” and “rumours”, though it was forced to suspend six of the alleged “vote fixers” named in the paper. Fifa’s Blatter has contrived to keep his job for 36 years, by means that would do credit to a Muammar Gaddafi. He professes “surprise” at talk of bribes while banning journalists like Jennings who ask questions about them from his press conferences…

Right on. Meanwhile I’m looking forward to a delightful three weeks in Provence while the London Olympics are on.

The amateur dictator

Posted on by

For years, I have wanted to use speech recognition software, but have always held back because the best product – Dragon Dictate – ran only under Windows, and I am a Mac user. But the company has released a version of the program for OS X, and I’ve just installed it. And this blog post is the first I’ve ever done simply by talking to the machine.

The accuracy of Dragon Dictate is scary. The strange thing is that we used to think that successful speech recognition was a problem for Artificial Intelligence. What none of us suspected was that it was a problem that could be solved using statistics and brute-force calculation. In fact, many years ago I knew someone in Cambridge – Frank Fallside – who was one of the pioneers of this approach to speech recognition. He died tragically young, so it’s nice to see how his work eventually came to fruition.

Android’s fragmentation problem

Posted on by

One of my boys has recently adopted my Android phone after his 6-year-old Motorola handset finally gave up the ghost, and it’s been interesting to observe his reactions. On the one hand, he’s charmed by finally having a handheld device that connects properly to the Net and the Web. But his experiences with Android Apps mirror mine, namely that there not much quality control, great variability and many Apps won’t work with lots of handsets. In fact, he’s experiencing the problems that finally drove me to get an iPhone.

What he hasn’t experienced yet, though, is the maddening control-freakery of the mobile carriers in relation to updating the OS on the handset. First of all, they accept no responsibility for the OS; and secondly, even when they grudgingly offer some upgrade facility, it’s often flaky and sometimes requires serious geek skills to implement. A friend of my daughter’s has the same Android handset (a t-mobile Pulse) and when I asked her what version of the OS it was running she said “I think it’s 2 point something”. Surprised (because she is not in the least geeky), I asked her how she’d done the upgrade from the version 1.5 that’s running on my handset. She replied that her brother — who is an engineering student and a real geek — does the upgrades for her. But then she added: “the only problem is that it crashes a bit after he’s done the upgrade”.

Dan Gillmor has an interesting piece in Salon.com in which he explores some of these issues.

The first problem, as I noted in a recent post, is that Google has given the mobile carriers nearly total control over the phones they sell — including the software. In the process, they’re taking Android — an open-source operating system when it gets to the carrier — and turning it into an operating system that removes user choice, by adding software that locks down the devices in ways that are even worse, in some respects, than the famous Apple control-freakery. At least Apple doesn’t load crapware — mostly unwanted, unneeded and un-removable software — onto the iPhone and iPad, as the carriers are doing with their Android devices. This has forced users to jailbreak their Android phones, a perversion of the very idea of openness.

We’ve seen the consequences of mixing manufacturer control-freakery with open source OSs already in the Netbook market, with every vendor offering its own infuriating version of Linux Lite. I’m tired of having to clear the disk of every Netbook I try in order to install Ubuntu. But at least the Ubuntu people take responsibility for their distribution, and they’re very helpful in relation to different brands of Netbook. Google should do the same for Android.

The iPadification of Mac OS

Posted on by

Hmmm… Apropos my column pondering the implications of iOS for Apple computers, here are some interesting thoughts on the same subject.

But it is the changes coming in Lion that are inspired by the iPad's user interface that will have broader ramifications for the future of all Macs, even desktops. These include the Launchpad screen and its folder-creation method, (OS-level support for) full-screen apps, auto-save and auto-resume. As with the iPad-inspired hardware changes, these will bring tradeoffs. Many of these make computing more accessible to newcomers, a path that Apple has doggedly pursued since the dawn of the Mac. To Apple's benefit, they also differentiate Mac OS further from Windows and tie together Apple's products better.

For veteran users, though, the changes may not represent an ideal execution. For example, auto-save can be a lifesaver, but for productivity applications it is ideally implemented with version control that is generally not in iPad apps today and which can be a confusing concept to new users. Similarly, the Launchpad interface may be effective for a world without mice or hierarchical folders, but Apple already offers the dock and the Applications folder for easily browsing programs. And with tried and true aids such as list view and sorting, one can take advantage of larger displays to view more apps at a glance without having to wander among screens, particularly when hunting for apps that are used less often.

But the hope is that Apple will blend them into the Mac OS rather than graft them on. Just as with the new MacBook Air, the key is to recognize what is relevant and what is not.

Apple hasn't yet offered extensive details on how these iPad calques will work in Lion; there doesn't seem to be any requirement for users to use these in Lion. But the hope is that Apple will blend them into the Mac OS rather than graft them on. Just as with the new MacBook Air, the key is to recognize what is relevant and what is not. For example, while Apple has dismissed physical keyboards on its iDevices, it continues to treat them as sacrosanct on the Mac, ensuring that its smallest notebook still has a keyboard with full-sized keys with spacing…

John Gruber has some interesting thoughts about this — as usual. For example:

iOS apps do run on Mac OS X, today, in the iPhone/iPad emulator that ships with the iOS developer kit. Ends up they’re just not that pleasant to use on a Mac. Gestures that are natural and fun with direct touch are awkward and clumsy using a mouse or touchpad. I never hear iPad developers — who run their own iOS apps on their Macs during development, for testing and debugging purposes — wish that they could ship them as-is to Mac users. Ever try a game like Pac-Man on the iPhone? A game that’s designed from the ground up around a hardware joystick or D-pad just isn’t very good on a device without a joystick. Everything about iOS apps is like that when you run them on a Mac. (And, conversely, popular iOS games like Angry Birds tend to feature controls that only really make sense with a touchscreen.)