To Corpus Christi for a CSaP lecture by Jaan Tallinn, Chief Engineer of Skype. Since he’s the Estonian programmer behind Kazaa (formerly the scourge of the music industry) and then a lead architect of Skype, I expect him to be talking about VoIP or some such geeky topic. He’s a big name in these circles and he plays to a packed house.

But it turns out that he doesn’t want to talk about geeky stuff and instead launches into a fascinating but wayward excursion into Kurzweil territory. He gets there via an unusual route, though: by arguing that, essentially, the human brain was evolution’s biggest mistake, because it has enabled us to divert the natural course of things with our infernal ingenuity — with potentially disastrous consequences. This is routine stuff for some audiences — for example those who share James Lovelock’s views about global warming. But it’s not CO2 emissions that bother Tallinn: it’s the ‘singularity’ that also obsesses Kurzweil. In other words, he extrapolates the increasing ‘intelligence’ and processing power of computers to the point where we will have created artificial intelligences that are smarter than us and which will have no further use for humans, save perhaps as pets. At which point I hear echoes of Bill Joy’s famous essay, “Why the Future Doesn’t Need Us”, and begin to wonder if this software wizard hasn’t, well, ventured into philosophical territory without even a rudimentary map.

But Tallinn is an entertaining speaker (and the only presenter I’ve ever seen who can actually use Prezi to good effect) so most of us temporarily suspend disbelief and stay connected. His central idea is of an “intelligence stairway” — a series of steps starting with self-replication leading to evolution leading to humans leading to tech progress leading to “artificial general intelligence” (AGI) and thence to an “intelligence explosion” which leads to the Kurzweil Singularity. Tallinn thinks (via reasoning that I can’t follow) that what follows next is “environmental catastrophe”. Is this because machines will be unconcerned about global warming, because they are capable of surviving it whereas organic life is not? Who knows? **See footnote**

The audience is intrigued but unconvinced. One attendee is sceptical to the point almost of derision: he doesn’t buy into Tallinn’s account of computational progress (which lays great stress on computers’ ability to play world-class chess, and he thinks that Tallinn’s citation of Apple’s SIRI as an illustration of how far computers have come in understanding people is way overblown. Another sceptic (I think an economist) takes the line that it’s difficult to see computers being able to understand context, and so the only we need to take in AI research is to make sure that they never do!

I am likewise entertained but unconvinced. But I am struck by one thought, namely that there are areas of scientific research where we do worry about a ‘stairway’ of the kind sketched by Tallinn — biotechnology and genetic engineering in particular, and also perhaps nanotechnology. Maybe we should do some thinking about the work that the 300 or so researchers working on AGI are doing? And is the reason why we don’t take the threat of AGI seriously the fact that, deep down, we simply can’t conceive of machines that are smarter than us? We have no problem envisaging scenarios in which, say, nanotechnology or genetically-modified organisms might run out of control and give rise to horrible unintended consequences. But computing machines…???

But it was an entertaining and thought-provoking lecture. On my way out through the throng of Cambridge academics and geeks engaging in the social activity quaintly known as “networking” I am suddenly struck by a vague memory from my past. I too once gave a lecture to a packed house. The audience appeared to love it and applauded loudly at the end. As I was leaving the theatre I noticed that one of my academic colleagues had been lounging at the back. “Very good lecture”, he said. “Just the right number half-truths”.

**Footnote

My colleague Anil Madhavapeddy was also there and writes:

“He [Tallinn] is falling into his own trap: any sufficiently advanced AI would
maintain itself until it can find a more algorithmically efficient source
of resources than earth (i.e., gas giants! space!) and would not work on
human timescales (whats the rush?).

On the other hand, one can imagine very easily a computing virus such as
Stuxnet II wiping out life on earth, due to it causing some cyberphysical
system to go ballistic and trigger something off by mistake. Not advanced
AI, just plain old insecure computer systems, and this does need fixing
urgently, and the GAI topic is an unfortunate distraction.”

After Adrian Monck had decamped from poorly-paid academic life to well-heeled employment in charge of Public Affairs for the World Economic Forum he once gently chided me for an intemperate, exasperated comment I had made about the Forum’s annual talkfest. In retrospect, I think I was reacting to the infuriating smugness of journalists (and bloggers) who were really just flaunting their entree into such an exclusive club — much as sports reporters might flaunt a ticket to the dining room at Augusta National during the Masters. But because I take Adrian seriously, I started paying more attention to what goes on in Davos. So I was intrigued to find that he has started blogging again with this thoughtful post. “Davos”, he writes.

“is an independently-minded moun­tain com­munity, steeped in Switzerland’s dir­ect demo­cratic tra­di­tion. Its alti­tude and an enter­pris­ing doc­tor, Alex­an­der Spen­gler, made it a des­tin­a­tion for well-heeled tuber­cu­losis suf­fer­ers. Thomas Mann set his com­edy of ennerv­a­tion, The Magic Moun­tain, in one of its san­at­oria. Albert Ein­stein helped kick-start its repu­ta­tion as an intel­lec­tual retreat (video).

Davos today is a work­ing alpine town. The town’s tour­ism is a func­tional con­trast to the chocol­ate box world of Vil­lars, Zer­matt and St Mor­itz. The Forum’s Annual Meet­ing boosts the local eco­nomy, but not its winter sports. Barely one-fifth of those par­ti­cip­at­ing can be accom­mod­ated in a five-star hotel. The local ski-lift com­pany has con­tem­plated shut­ting the lifts dur­ing the Meet­ing. When I’m there, as a mem­ber of the Forum, I sleep on a single bed and share a bath­room. Hard­ship? Not really, but it is work.

And that suits the Forum, because it deals with the world as it is, not as it would prefer it to be. It is not a decision-making body. Nor is it a con­spir­acy in which the horo­lo­gical com­pon­ents of global gov­ernance and industry are wound together to frus­trate the rest of the world…”.

Of course one has to remember that he works for the forum, so the Mandy Rice-Davies caveat applies. Still, the next post on the blog gives a set of interesting, contradictory and sometimes critical perspectives on this year’s event.

I’ve been to Davos once — many years ago, long before the Forum was thought of. I thought it a rather dull, workaday place, quite different from St Moritz which is just around the corner. I remember it chiefly because I bought a Swiss Army knife there which I still have.