Behind the Valukas report into Lehman Brothers

The 2,200-page report into the collapse of Lehman Brothers carried out by Anton Vulakas at the request of the New York Southern District Bankruptcy Court will, no doubt, make interesting reading, but what caught my eye was this sentence in the Financial Times‘s report:

“The crux of the report, which is based on the review of 34 million pages of documents out of the 350 billion pages obtained by Mr Valukas, is its portrayal of Lehman’s insatiable risk appetite and its alleged efforts to cover up the extent of its financial woes.”

How on earth, I wondered, does anyone ‘review’ 34 million pages? And how are those pages selected from 350 billion? Not surprisingly, I wasn’t the only one asking these questions. Writing in The Posse List, Gregory Bufithis provided some of the answers. “The most intriguing part of the report”, he writes,

“concerns the sheer size of the data and the search methodology/software used in examining the documents. Valukus’s report was a mammoth task involving e-mails, reports, data sets and interviews. Answering the questions required an extensive investigation and review of Lehman’s operating, trading, valuation, financial, accounting and other data systems. Interrogating those systems proved particularly challenging, first because the vast majority of the systems had been transferred and were under the control of Barclays (who took over a large part of Lehman operating units); by the time of the Valukas’ appointment, Barclays had integrated its own proprietary and confidential data into some of the systems, so Barclays had legitimate concerns about granting access to those systems.

The second challenge was more daunting. At the time of its bankruptcy filing, Lehman maintained a patchwork of over 2,600 software systems and applications. It was decided early on that it would not be cost effective to undertake the enormous effort and expense that would be required to learn and access each of these 2,600 systems. Rather, Valukas directed his financial advisors to identify and acquire an in‐depth understanding of the most promising of the systems.”

Mr Bufithis then goes on to outline how this mammoth task was tackled:

1. The available universe of Lehman e‐mail and other electronically stored documents is estimated at three petabytes of data — roughly the equivalent of 350 billion pages.
2. Valukus carefully selected a group of document custodians and search terms designed to cull out the most promising subset of Lehman electronic materials for review. In addition, Valukus requested and received hard copy documents from Lehman and both electronic and hard copy documents from numerous third parties and government agencies.
3. In total, the Examiner collected in excess of five million documents, estimated to comprise more than 40,000,000 pages. All of these documents have been converted to electronic form and are maintained on two computerized databases, Stratify and CaseLogistix.
4. Documents were reviewed on at least two levels. First level review was conducted by lawyers trained to identify documents of possible interest and to code the substantive areas to which the documents pertained; those so identified were subjected to further and more careful review by lawyers or financial advisors especially immersed in the earmarked subjects. In order to reduce the cost of review, the Valukus sought and obtained the court’s approval to retain contract attorneys. A group of more than 70 contract attorneys, supplemented by Jenner & Block [Valukas’s Chicago law firm] attorneys, conducted first level reviews.
5. All second level (and beyond) reviews were performed by Jenner & Block attorneys or Duff & Phelps professionals. Valukus estimates that he has reviewed approximately 34,000,000 pages of documents in the course of his investigation.
6. The entire body of e-mail in the Stratify database — 4,439,924 documents, approximately 26 million pages— has been reviewed. Approximately 340,000 of the CaseLogistix documents — roughly eight million pages — have been reviewed.
7. Although a large number of the CaseLogistix documents were not reviewed, that database is fully searchable, and Valukus is reasonably confident that the repeated and focused searches applied against that database have discovered most if not all of the most relevant documents.
8. In most cases, documents were produced to Valukus under stipulated protective orders, which are described in Appendix 5 of the report. Subject to those orders, the document databases created remain a resource for the bankruptcy estate and the parties. The database includes computerized tagging which will allow persons interested in making their own searches to narrow and focus search requests.

And the cost of all this? A snip at $38 million. Wonder who pays the bill?

The Facebook question

This morning’s Observer column:

Is Facebook now “too big to fail”? I don’t mean in the sense that the taxpayer would have to pick up the pieces if it went under, but in the sense that the social networking service has achieved a position of such dominance in the online ecosystem that its eclipse is unthinkable. Is Facebook, in other words, the next Microsoft or Google?

The question is prompted by a couple of milestones recently passed by Facebook. The first is that it now has more than 400 million members. The second is industry gossip predicting that its revenues for 2010 will exceed a billion dollars. Other straws in the wind are estimates of the size of the “Facebook economy” – ie the ecosystem of applications, services and products that has evolved around the service; and the moral panics it now triggers in the mainstream media – a sure sign that they fear a competitor…

Chatroulette and common sense

The moral panic du jour is about Chatroulette, and it’s tiresome. When reading some of the commentary that’s mushroomed around it I suddenly had the urge to see what danah boyd had to say about it, partly because she’s thought more profoundly about social networking than anyone else, and partly because she’s a rock of common sense in these matters. And, sure enough, she has a thoughtful post on her blog. Excerpt:

What I like most about the site is the fact that there’s only so much you can hide. This isn’t a place where police officers can pretend to be teen girls. This isn’t a place where you feel forced to stick around; you can move on and no one will know the difference. If someone doesn’t strike your fancy, move on. And on. And on.

I love the way that it mixes things up. For most users of all ages – but especially teens – the Internet today is about socializing with people you already know. But I used to love the randomness of the Internet. I can’t tell you how formative it was for me to grow up talking to all sorts of random people online. So I feel pretty depressed every time I watch people flip out about the dangers of talking to strangers. Strangers helped me become who I was. Strangers taught me about a different world than what I knew in my small town. Strangers allowed me to see from a different perspective. Strangers introduced me to academia, gender theory, Ivy League colleges, the politics of war, etc. So I hate how we vilify all strangers as inherently bad. Did I meet some sketchballs on the Internet when I was a teen? DEFINITELY. They were weird; I moved on. And it used to be a lot harder to move on when everything was attached to an email that was paid for. So I actually think that the ChatRoulette version allows you to move on with greater ease, less guilt, and far more comfortably. Ironically – given the recent media coverage – it feels a lot safer than any site that I’ve seen that’s attached to a name or profile with connections to people or identifying information. Can youth get themselves into trouble here? Sure… like in most public places. And there are definitely youth who are playing with fire. But, once again, why go after the technology when the underlying issues should be the ones we address? Le sigh.

There’s also an interesting interview in the New York Times with the kid who created ChatRoulette. What caught my attention is this excerpt.

Why did you start Chatroulette?
I was looking for a site like this, one that would let me chat randomly on webcams, and I couldn’t find it, so I thought I would try to build it.

How long did it take to build?
It took me three days. I built it on an old computer I had in my bedroom.

Then what happened?
Well, at first I showed it to my friends and they criticized it; they asked why anyone would want to use it. So I went onto a few Web forums and asked people to try the site, and I got 20 people to try it.

How many users do you have now?
Well, after the initial 20 users the site doubled and it continued to double every day since then. Last month I saw 30 million unique visitors come to the Web site and one million new people visit each day. It continues to multiply and I just couldn’t stop it from growing.

What were you thinking while this was happening?
I woke up one morning and checked my computer and saw all of these news articles about Chatroulette. I yelled to Mom to come and look at my computer. At first she was very nervous, but she doesn’t really understand it very well and asked me why I’m not going to school.

This resonates because I’m working on a book at the moment which is largely about how mainstream culture still doesn’t understand the essence of the Net. I’m arguing that a useful way to think about it is as a global machine for generating surprises. The Web was one such surprise; Napster was another; malware yet another. Chatroulette is a surprise in the same tradition: a smart idea implemented by a smart kid, at virtually light speed, using an old PC and in his bedroom! And without having to ask anyone’s permission. It’s an example of the explosive creativity enabled by the architecture. No wonder the Daily Mail (and New Labour) has such a hard time comprehending it.

Libraries and the digital record

Jonathan Zittrain from the Berkman Center at Harvard gave this riveting lecture at Duke University on March 3. It’s quite long — an hour and a quarter — so you need to allocate some serious time to it, but IMHO it’s worth it. It starts slowly as he lays out an analytical framework that, at first sight, seems to have little to do with libraries, but about 27 minutes in to the presentation he really hits his stride. For anyone interested in the cultural responsibilities of libraries in a digital era, this is eye-opening stuff becasue it gives some concrete examples of cases where libraries will need to assume really serious responsibilities as curators of the digital record, not just in terms of preservation, but also in defence of historical accuracy.

Toward A New Alexandria

Long, thoughtful and wide-ranging article by Lisbet Rausing in The New Republic about the future of academic libraries in a digital environment.

It is clear that if a new Alexandria is to be built, it needs to be built for the long term, with an unwavering commitment to archival preservation and the public good. A true public good itself, it probably needs to be largely governmentally funded. And, while a global and cooperative venture, it needs to be hosted by one organisation that is reputable, long-standing, nonprofit, and exists in a stable jurisdiction. The Library of Congress, the flagship institution of the world’s only surviving Enlightenment republic, comes to mind. There might be other possibilities, such as the New York Public Library, or the British Library, or a consortium of the world’s leading university libraries—UCLA, Harvard, Cambridge University, and so on.

In other words, the question for scholars and gatekeepers is not whether change is coming. It is whether they will be among the change-makers. And if not them, then who? Who else will ensure long-term conservation and search abilities that are compatible across the bibliome and over time? Who else will ensure equality of access? Ultimately, this is not a challenge of technology, finances, or ultimately even laws, difficult though they are. It is a challenge of will and imagination.

Answering that challenge will require some soul-searching: Do we have the generosity to collaborate? Can we build legal, organizational, and financial structures that will preserve and order—but also share and disseminate the learning of the world? Scholars have traditionally gated and protected knowledge, yet also shared and distributed it in libraries, schools, and universities. We have stood for a republic of learning that is wider than the ivory tower, and now is the time to do so again. We must stand up, as the Swedes say, for folkbildningsidealet, that profoundly democratic vision of universal learning and education…

Worth reading in full.

Waiting for the iPad

The iPad is coming (beginning of April in the US, late April in the UK) but nobody has the faintest idea what will happen when it arrives. All over the computing industry, manufacturers are frantically trying to get their own iPad-lookalikes ready. HP has got one, apparently. No doubt ASUS has too. Google is rumoured to be working on an Android slate. And so on. Up to now, there’s only been a niche market for tablet devices (despite Bill Gates’s historic conviction that they would be the New Big Thing.) The $64 billion question is whether the Apple product will rescue the industry by creating a whole new product category — between the netbook and the laptop/desktop.

The content industry — especially the publishers of high-end magazines — is also waiting with bated breath to see what happens. Will the device rescue print from having to go down the cul-de-sac of web paywalls? That’s why publishers are so interested in putting out their publications as Apps rather than sites: doing it that way means that there’s a way of charging for content that consumers apparently find acceptable. The problem, of course, is that that gives Apple another chokehold on online content: everything has to go through the iTunes store, and Apple gets a cut of all the action thereon.

So this is a strange time: a huge industry is holding its collective breath to see if a single company will change everything. Will the iPad be a game-changer, as the iPhone has proved to be? Or will people buy it and then wonder — after the novelty has worn off — if it was worth all the fuss? Nobody knows.

LATER: It seems that Apple is barring UK customers from pre-ordering the iPad.

You could make it up — but who’d believe it?

Lovely story from Sean & Nicci’s blog.

J.P.R. Williams, a celebrated Rugby international from my youth and now a 61-year-old orthopaedic surgeon, is stopped by police and breathalysed. But he has a bright idea. Someone, somewhere, has told him that you can cheat the test by putting copper coins in your mouth. The problem is that it's an idea that you need to be drunk to believe. The further problem is that PC Plod tends to get suspicious if you start shoving small change into your mouth. The even further problem is that it doesn't work.

He's been banned from driving for seventeen months. They should also make him take a GCSE in Chemistry.

Reading this scene in the manuscipt of a novel, an editor would surely object: funny idea, but do we have to make him a doctor? Nobody is going to believe that.

If only J.P.R. had read the entry on the subject in the great urban legends website, snopes.com.

Digital-to-analogue, 2010

Hmmm… Just when I’ve been experimenting with escaping from the tyranny of ‘perfect’ digital imagery, I find that one of my sons has been heading off in the same direction. Except that he’s been doing it the hard way. “As a counter-balance to the increasingly digitised world of photography”, he writes, “I have decided to explore the analogue printing methods of yesteryear. Specifically I am focusing on screen printing, a process that was famously popularised by Andy Warhol.”

In true Warhol spirit, his first experiments have focussed on a quintessential British brand — Marmite! What’s nice about them is that no two prints are the same, which in a way is like an implicit definition of analogue media. A gallery of images of the resulting work can be found here, and a downloadable flyer [pdf] here.

Now, where did I put that tomato ketchup?

Later: You think I jest about Marmite being quintessentially British? An English journalist friend lived and worked in New York for a couple of years and he swore that the thing he missed most in all that time was… Marmite.

Facebook bites back

From today’s Guardian.

Facebook has threatened to sue the Daily Mail for damages after the paper wrongly claimed in a piece published on Wednesday that 14-year-old girls who create a profile on the social networking site could be approached “within seconds” by older men who “wanted to perform a sex act” in front of them.

The paper apologised in print today and online yesterday for the error, which the author of the piece, Mark Williams-Thomas, insisted had been introduced by editors at the paper despite being told it was wrong. In fact, Williams-Thomas – a retired policeman who now works as a criminologist – had been using another, unspecified social network.

But the giant social networking site, which has 23 million users in the UK alone, said that although the Mail has changed the headline of the article online – so that it now reads “I posed as a girl of 14 online. What followed will sicken you” – it had not at first changed the page title of the article online, used by internet search engines to index content, nor the URL of the piece, which is also a factor in search-engine indexing.