“The New Research Seeks To Mine Web Data To Predict All Kinds Of Events”

Marshall McLuhan wondered how the new environment would be programmed in the Digital Age, but here’s another important question: How will use our new access to predict the future? Researchers are currently studying decades of newspaper archives in an effort to protect us from dangers in the queue. From Gigaom:

“Researchers at Microsoft and the Technion-Israel Institute of Technology are creating software that analyzes 22 years of New York Times archives, Wikipedia and about 90 other web resources to predict future disease outbreaks, riots and deaths — and hopefully prevent them.

The new research is the latest in a number of similar initiatives that seek to mine web data to predict all kinds of events. Recorded Future, for instance, analyzes news, blogs and social media to ‘help identify predictive signals’for a variety of industries, including financial services and defense. Researchers are also using Twitter and Google to track flu outbreaks.

Eric Horvitz of Microsoft Research and Kira Radinsky of the Technion-Israel Institute describe their work in a newly released paper, ‘Mining the Web to Predict Future Events (PDF). For example, they examined the way that news about natural disasters like storms and droughts could be used to predict cholera outbreaks in Angola. Following those weather events, ‘alerts about a downstream risk of cholera could have been issued nearly a year in advance,’ they write.

Horvitz and Radinsky acknowledge that epidemiologists look at some of the same relationships, but ‘such studies are typically few in number, employ heuristic assessments, and are frequently retrospective analyses, rather than aimed at generating predictions for guiding near-term action.’”