Incomprehensible computer behaviors have evolved out of high-frequency stock trading, and humans aren't sure why. Eventually, it could start affecting high-tech warfare, too. We spoke with a researcher at University of Miami who thinks humans will be outpaced by a new "machine ecology."
For all intents and purposes, this genesis of this new world began in 2006 with the introduction of legislation which made high frequency stock trading a viable option. This form of rapid-fire trading involves algorithms, or bots, that can make decisions on the order of milliseconds (ms). By contrast, it takes a human at least one full second to both recognize and react to potential danger. Consequently, humans are progressively being left out of the trading loop.
And indeed, it’s a realm that’s rapidly expanding. For example, a new dedicated transatlantic cable is being built between US and UK traders that could boost transaction speed by another 5 ms. In addition, the new purpose-built chip iX-eCute is being launched which can prepare trades in an astounding 740 nanoseconds.
The problem, however, is that this new digital environment features agents that are not only making decisions faster than we can comprehend, they are also making decisions in a way that defies traditional theories of finance. In other words, it has taken on the form of a machine ecology — one that includes virtual predators and prey.
Consequently, computer scientists are taking an ecological perspective by looking at the new environment in terms of a competitive population of adaptive trading agents.
“Even though each trading algorithm/robot is out to gain a profit at the expense of any other, and hence act as a predator, any algorithm which is trading has a market impact and hence can become noticeable to other algorithms,” said Neil Johnson, a professor of physics at the College of Arts and Sciences at the University of Miami (UM) and lead author of the new study. “So although they are all predators, some can then become the prey of other algorithms depending on the conditions. Just like animal predators can also fall prey to each other.”
When there’s a normal combination of prey and predators, he says, everything is in balance. But once predators are introduced that are too fast, they create extreme events.
"What we see with the new ultrafast computer algorithms is predatory trading,” he says. “In this case, the predator acts before the prey even knows it's there."
Johnson describes this new ecology as one consisting of mobs of ultrafast bots that frequently overwhelm the system. When events last less than a second, the financial world transitions to a new one inhabited by packs of aggressively trading algorithms.
Now, while we’ve known about high-frequency stock trading for years now, what’s less known is the frequency of ultrafast extreme events (UEEs). In the context of stock trading, a UEE (sometimes referred to as a flash freeze) manifests as a crash or spike. That is, an event in which a stock price ticks down or up at least ten consecutive times before changing direction, and the price change exceeds 0.8% of the initial price.
According to Johnson’s research, there were a jaw-dropping 18,520 crashes and spikes with durations less than 1,500 ms between January 2006 and February 2011.
And disturbingly, as the duration of these UEEs fell below human response times, the number of crashes and spikes increased dramatically. What’s more, these crashes could not be attributed to other factors.
Crashes are problematic on their own, but they’re doubly so when we’re left out of the loop. Without human oversight, these algorithms battle it out to take control.
“It simply is faster than human predators (i.e. human traders) and the humans are inactive on that fast timescale,” says Johnson. “So the only active traders at subsecond timescales are all robots. So they compete against each other, and their collective actions define the movements in the market.”
In other words, they control the market movements. “Humans become inert and ineffective,” he says, “What we found, which is so surprising, is that the transition to the new ultrafast robotic ecology is so abrupt and strong.”
There’s also the global economy to consider. UEEs can result in market instability, highlighting the need to develop countermeasures — realtime interventions that could mitigate risk.
“There is real money being gained and lost here — even a few thousand dollars every millisecond, which is a tiny amount on the market, is a million dollars per second,” he told us. “This money could be pension fund money, and so on. So somebody needs to understand what is going on, and if it is 'fair'.”
If we don’t understand what’s going on, he argues, it will be impossible to make informed decisions as to whether regulation is required, and if so then what type.
“In terms of risk to the financial system as a whole, the fact that our paper reports heightened numbers of extreme events within financial stock prior to the 2008 financial crash, strongly suggests that there is indeed a connection between what goes on in this subsecond world, and the usual financial market timescales of days and weeks,” he says. “Most of us have our pensions buried in the markets, so in addition to pure science interest, this issue is of direct relevance to everyone.”
We asked Johnson why we can’t just slow down the process and legislate human-comprehensible timescales.
“Well, there will always be companies trying to go faster,” he responded. “If there was a minimum time for the transaction, then it would be like holding people at the door on the first day of a sale, then opening the doors to a stampede. The Europeans are trying to apply a so-called Tobin tax on transactions. But lawyers today have said this may be illegal, so it is a complex situation concerning what to do — in part because no one fully understands what is actually going on. We have only scraped the surface in our paper, but we are confident there are many strange effects waiting to be discovered in the subsecond trading world.”
Indeed, Johnson worries that this new all-machine phase could extend to the world of cyber-attacks and cyber-warfare.
Given the presence of rival agents, bots could be set against each other in a similar manner, again creating a digital ecology outside of our control and comprehension. But unlike a market crash, a UEE in this context could result in something quite catastrophic, including the collapse of the IT or energy infrastructure. It could even give rise to an advanced artificial intelligence.
Another possible solution, albeit a more radical one, is that we engage in human intelligence amplification. By augmenting human intelligence, we may be able to stay in the loop. Modified humans could enter into this new digital jungle and take part as a generalized intelligence in an effort to regain control. Trouble is, IA could result in severe downstream psychological consequences, such as insanity and psychopathy. Alternately, we could develop a specialized AI or AGI (artificial general intelligence) to perform this work on our behalf.
More simply, Johnson says we need to better understand the collective behavior of these interacting systems if we’re to avoid problems like microcrashes. Surprisingly, he says this may not be as difficult a proposition as it sounds.
“Algorithms are ultimately deterministic, even though they might try to act 'randomly' to avoid detection,” he explains. “So while humans have 'free will' and can have all sorts of strange and random behaviors, algorithms can be doing simple things like predicting that the next movement will be a continuation of the previous few timesteps.”
The trouble, he says, is that there may be many robots that behave similarly, since this particular algorithm is an obvious but simple one, which means that crowds/mobs of these algorithms spontaneously form and act.
What’s more, if cyber arms races are to continue apace, the sophistication of these bots will only increase. As time passes, the complexity of these new ecologies may prohibit the kind of safety measures Johnson calls for.
But looking ahead, Johnson is optimistic about our chances.
“We feel we have already made an advance in our present paper,” he told io9. “Though it is not yet the full story, our paper sets the scene for a systematic effort to unravel the various processes at play.
According to Johnson’s paper, which now appears in Nature Scientific Reports, every UEE is the result of a sudden excess buy or sell demand in the market. Their new model provides a simple explanation for how these sudden excess buy or sell demands are generated (but not how they get fulfilled).
“Even with the results we have so far, in particular our mathematical ecological model, we can check 'what if' scenarios about the future, and also attempt to reverse engineer what is going on when one of these ultrafast 'accidents' happen.”
Read the entire study in Nature Scientific Reports: “Abrupt rise of new machine ecology beyond human response time.”