Someone like Bob could tell more about the direction of the markets or the future of geopolitics from a mere glance at a stock ticker or a read-through of a newspaper than most of us could from a year of study on the internet. This is because there is more to forecasting than mere data. It requires experience, discernment, and gut instinct, as well as decades of refinement of these faculties.
When Bob Chapman started this publication in 1991, only a tiny percentage of the public had ever sent or received an email and the world's first website had only been created a few months before. Now, just over two decades later we send and receive dozens (or hundreds) of emails a day from our smartphones, all while scrolling through our Twitter feed, posting to our personal blog, checking our bank account, ordering items on eBay, getting Siri to book a table at our favorite restaurant and playing Candy Crush. It's often remarked that the pace of technological change is increasing, but I wonder if we appreciate what that really means.
As a man in my mid-thirties I straddle the internet divide, having spent about half of my life with the internet and half without. I am young enough to have never worked in an office without email and old enough to have never had an email account in grade school. From this vantage point, the internet is a mundane fact of life, something that I interface with nearly every day of my life. But from time to time I am able to reflect on what life was like before the internet and the massive transformation of our society that it has brought (and is bringing) about.
This newsletter is just one reflection of that. Within the space of a few hours, I can read, process, collate and cross-reference news stories from virtually anywhere in the world and place them in an historical context gleaned from any number of web sites and digitized media. At my fingertips at literally any time of the day or night is access to more information than Bob Chapman could have dreamed of in 1991. This access makes me (and virtually anyone else) an “expert” on any field of study we decide to delve into, merely by virtue of having the collective wisdom of thousands of experts at our beck and call.
It is easy to let this access to information go to our heads, and for us to believe ourselves to be smarter than those who went before us. But there is a big difference between information and knowledge, and we would do well to remember that. Someone like Bob could tell more about the direction of the markets or the future of geopolitics from a mere glance at a stock ticker or a read-through of a newspaper than most of us could from a year of study on the internet. This is because there is more to forecasting than mere data. It requires experience, discernment, and gut instinct, as well as decades of refinement of these faculties.
But don't tell that to the planners at the Pentagon or those working in the bowels of the military-industrial-information complex. In the increasingly digitized world of cyberwarfare and keyboard warriors, raw data alone is the key to unlocking all the secrets of the future. Following Archimedes' famous boast, “Give me a lever long enough and a fulcrum on which to place it, and I shall move the world,” the mantra of the NSA/Pentagon types might be “Give me enough data and a data center to store and process it, and I shall predict the future.” Sound fanciful? Not at all. They're already trying to do it.
In 2002 DARPA created the Information Awareness Office under the command of convicted Iran-Contra criminal Admiral John Poindexter. Its aim was to produce “Total Information Awareness” via the collection and assimilation of every piece of data the government could get its hands on: financial data, education records, travel history, communications data, medical records, even veterinary records. This was to be compiled in order to give the US federal government and its agencies unprecedented access to information on its citizens, information that could be used to identify, track, trace, catalogue, database, relate and even predict the future movements or actions of any given individual. This was extrapolated into a program called Future Markets Applied to Prediction, or FutureMAP, which would allow users to “bet” on future political events, such as when and where the next terror attack would occur. It was proposed that this technique would harness the predictive power of the markets to help the government more accurately predict political instability and threats to national security, but when it was brought to the attention of the Senate it was roundly dismissed as “ridiculous” and “grotesque” and abandoned.
In the early part of last decade, researchers at Purdue University, funded by the Department of Defense, developed a program called Synthetic Environment for Analysis and Simulations, or SEAS. The program uses breaking news, economic data, census figures, military intelligence, climate data, and other information to produce realistic simulations of real-world geopolitical events. The end goal of the researchers is to create a complete simulation of the entire world, called the Sentient World Simulation, that will contain billions of “nodes” representing every single person on the planet, and which will be able to predict their individual and collective reactions to various events. The US Joint Forces Command has been using a version of the program to carry out simulations and war games for over half a decade now.
In 2008 the British press reported on CCTV cameras that were able to independently monitor suspicious activity in a given area and alert authorities to potential problems before they even occur. The technology, developed and marketed by a company called Smart CCTV, allows cameras to distinguish between people, vehicles and background objects, and to examine those entities for suspicious activity, like cars that are circling past the same area repeatedly. The cameras could inform authorities of that suspicious activity before a crime even takes place. At the time of the initial reports, the company had already inked a deal with Portsmouth's local council, and was in talks with 10 other local authorities to install the cameras for police use.
In 2009 a team of researchers concluded that data collected from cell phones can be used to predict the movements of those phones' owners to a 93% accuracy rate. The researchers used cell phone location data culled from phone company records to examine the movements of a number of cell phone users and discovered that by examining the movements of people through their daily routine, they were able to accurately predict where someone would be at any given time as often as 97% of the time. In 2013, leaks from NSA whistleblower Edward Snowden confirmed that the NSA has been collecting the so-called “metadata” of phone calls from cell phone users for some time now, just the sort of information that could be used to predict the future movements of those individuals.
Later in 2009, IBM acquired a statistical analytics program called SPSS and a related data mining application called SPSS Modeler. The program can take information from a number of data sources, including text databases and social network information, and create predictive models on any number of events and situations. One of the premier applications of this technology is for use in what IBM calls “predictive policing,” which enables cities to develop predictive models for criminal activity. According to IBM, this technology is already being used in Memphis, Richmond, Lancaster and even Incheon, South Korea to help reduce crime.
In 2012 the New York Times published an article outlining how American retail giant Target started a program to target ads at women who were between 14 and 25 weeks pregnant, as this is the time at which women are most likely to change their shopping habits. In order to accomplish this targeted advertising, they began analyzing the raw data collected through their “Guest ID” program, which assigns each customer a number that is tied to their email address, name or credit card number. Eventually, they were able to develop an algorithm which used information about the purchases of 25 specific items to assign each customer a pregnancy score, and customers that hit a certain threshold were sent a package of coupons related to baby products. As a result, Target's mother and baby product revenue surged and overall revenue went from $44 billion to $67 billion.
Also in 2012 a secret surveillance system known as “Trapwire” was exposed by the email hack of Austin, Texas-based private intelligence company, Stratfor. It collects CCTV data from a network of cameras in places like New York, Washington, L.A., London and Ottawa, analyzing that information for suspicious behaviour and sharing it across the network in an effort to predict and prevent terrorist attacks on high-profile targets.
If all of this sounds creepy, then congratulations; you've got your head screwed on straight. That government agencies and associated researchers have been so focused over the past decade in predicting what you are going to do before you do it should make you uneasy. After all, we should keep in mind that in more cases than not the real criminals are in government itself, so this predictive technology in their hands is particularly worrying. But if all this information is making you panic, fret not; there are indications that a lot of it is hype that is being deliberately pumped up to make the public believe that the government is more powerful than it really is.
According to a story by Noah Schachtman in Wired's “Danger Room” in 2011, the DoD handed out $90 million between 2008-2011 on over 50 research labs to develop basic tools, theories and processes for developing reliable prediction technologies for use in spotting geopolitical events. The results of all that research have been mixed at best. In fact, as Schachtman goes on to point out, the best of the bunch, a Darpa-funded program called Integrated Crisis Early Warning System, of ICEWS, was only able to predict 4 of the previous 16 geopolitical rebellions or political upheavals. In other words, monkeys with darts would be a more reliable prediction method.
Which brings us back to where we started. In the end, accurate prediction is not about technology. Any given piece of software is only as good as the person who programmed it. Which is why the insights of the Bob Chapmans of the world (not that anyone can quite replace Bob) are so important. It takes a real person with real experience to process the thousands of pieces of data that it takes to accurately forecast a market move or a geopolitical event.
Here's hoping that the NSA and DHS and DARPA and the rest of the alphabet soup don't catch on to this fact anytime soon.