Son of paperclip
Mar 22nd 2001
From The Economist print edition


Mobile Manager has an almost telepathic ability to distinguish junk mail from genuinely important messages

ERIC HORVITZ, a researcher at Microsoft and a guru in the field of Bayesian statistics, feels bad about the paperclip-but he hopes his latest creation will make up for it. The paperclip in question, as even casual users of Microsoft's Office software will be aware, is a cheery character who pops up on the screen to offer advice on writing a letter or formatting a spreadsheet. That was the idea, anyway. But many people regard the paperclip as annoyingly over-enthusiastic, since it appears without warning and gets in the way.

To be fair, that is not Dr Horvitz's fault. Originally, he programmed the paperclip to use Bayesian decision-making techniques both to determine when to pop up, and to decide what advice to offer. (Bayesian statistics are named after Thomas Bayes, an 18th-century Presbyterian minister who devised a mathematical rule for explaining how you should change your existing beliefs in the light of new evidence.)

The paperclip's problem is that the algorithm (sequence of programming steps) that determined when it should appear was deemed too cautious. To make the feature more prominent, a cruder non-Bayesian algorithm was substituted in the final product, so the paperclip would pop up more often.

Ever since, Dr Horvitz has wondered whether he should have fought harder to keep the original algorithm. The impressive abilities of his latest creation, an e-mail filtering system called Mobile Manager, suggest that he should have. In essence, Mobile Manager evaluates incoming e-mails on a user's PC and decides which are important enough to forward to a pager, mobile phone or other e-mail address. Its Bayesian innards give it an almost telepathic ability to distinguish junk mail from genuinely important messages.

Mobile Manager grew out of a two-year-old research system called Priorities, a more powerful filtering system that Dr Horvitz developed for his own use and is now widely employed internally at Microsoft (a practice known as "eating your own dogfood"). Priorities takes a vast amount of information into account when evaluating the importance of an incoming message. For a start, it looks at the message itself to see if it contains urgent language (such as "tomorrow" or "as soon as possible"), direct questions, the recipient's name, and statements in the future (as opposed to the past) tense. It checks to see if the sender is in the recipient's address book, whether the message has been "carbon copied" to many users, whether the message is short or long, and what the message's internal structure is. It then combines these and other clues using Bayesian decision theory to evaluate the "cost of delayed review per hour"-in other words, the message's importance.

But the cleverness does not end there. Over time, Priorities learns what a particular user regards as important or unimportant. Messages that are deleted without being opened are taken as examples of unimportant messages. Messages that the user replies to immediately are regarded as important. The software can also be trained explicitly by sorting a handful of messages into "important" and "unimportant" categories.

The result is a constantly updated list of incoming e-mails ranked by urgency. But Priorities also decides what to do with each message, by taking into account the context of its arrival. It can look at the user's calendar to see whether a meeting is going on, or whether he is on holiday. It can use a camera to determine if the user is sitting in front of a PC and whether he is looking at the screen. It can analyse ambient sound to determine whether he is having a conversation, and hence whether the user is busy or not. These factors are employed to decide whether to sound a chime, display a visual alert or, if the user has been away from the PC for more than a few minutes, to send a message to a pager or other device.

The software also allows the user to specify how important messages have to be to trigger mobile alerts. A message's importance, however, is not fixed. A message that is important on a quiet Thursday morning may not be so vital as to interrupt Sunday lunch or a vital meeting. Alerts can also be delivered in batches at specified intervals, or when a certain number of important messages have accumulated.

Mobile Manager, which offers a subset of Priorities' features as an add-on for Microsoft Outlook, was released on the company's website in February. Priorities' more Big Brother-ish features, such as watching and listening to users, are not included. But the Bayesian decision-making system is. Thanks to the paperclip, this technology got an undeservedly bad name for intrusiveness. This time around, the same technology is intended to do the opposite, and bother people less. It is a fitting antidote to the pesky paperclip.