After the "flash crash" that brought the stock exchange to its knees a few months ago, data analysis firm Nanex started looking into some odd patterns in stock trades. They discovered bot armies are affecting trades on a microsecond-by-microsecond basis.
Over at The Atlantic, Alexis Madrigal has a fascinating story about how Nanex software engineer Jeffrey Donovan started analyzing market data in a way nobody had ever done before. Instead of looking at trades and quotes served up each second or minute, he delved deeper. He looked for market patterns at the sub-second level, and discovered some seriously bizarre things were happening. Trading bots, controlled by unknown algorithms, were doing things like requesting stock quotes at a rate of 56,000 per second.
So are these algorithms created by malicious hackers, trying to recreate the Flash Crash? That is one possibility, since sometimes they request so much data that human trading could be affected. What seems certain is that they're not traces from the bots used by high-frequency traders, who often use algorithms to figure out where to buy and sell quickly and constantly. These bots' information requests and trades are just too random to be useful for high-frequency traders.
The algorithms . . . don't serve any function in the market. University of Pennsylvania finance professor, Michael Kearns, a specialist in algorithmic trading, called the patterns "curious," and noted that it wasn't immediately apparent what such order placement strategies might do.
Donovan thinks that the odd algorithms are just a way of introducing noise into the works. Other firms have to deal with that noise, but the originating entity can easily filter it out because they know what they did. Perhaps that gives them an advantage of some milliseconds. In the highly competitive and fast HFT world, where even one's physical proximity to a stock exchange matters, market players could be looking for any advantage.
"They are moving the high-frequency services as close to the exchanges as possible because even the speed of light matters," in such a competitive market, said Stanford finance professor Peter Hansen.
Donovan calls the visualizations of the data he found "crop circles," and he's created a blog where he posts a daily chart of unexplained bot activity. He says he finds several examples of weird algorithmic behavior at the sub-second level every day, so obviously the market service computers are swarming with bot activity.
The question remains: Who is controlling these bots? What exactly are they trying to do? Some analysts dismiss these nanosecond patterns as simple chance, the emergent properties of a complex system. Others, like Donovan, are just trying to gather data and understand them. But I think you and I know exactly what's going on. Out of the tiny impulses of a million bots, an artificial intelligence is being born.
This is an extreme closeup of just one second of trading of the stock SHG, the Shinhan Financial Group. This is 760 quotes from a total of 10,000 made in 12 seconds.
Here we see a "flag repeater" being executed on the BATS Exchange, the third-largest equity market after the NYSE and NASDAQ. 15,000 quote requests were made in 11 seconds in a repeating pattern. Each iteration upped the quote a penny until $9.36, and then the algorithm went down the same way, a penny at a time.
This chart shows a different kind of strategy. It represents 56,000 quotes in one second all at the same price (the top chart) but with the size of the order increasing by one (i.e. 100 shares) all the way up to 40,000.
Finally, we see what Donovan calls the "stubby triangles" chart. It shows high quotes being made and then immediately followed by a stub order of $0.01 (basically canceled in most contexts). The quote is then remade at a lower price and followed with another stub quote. This cycle happened at the rate of 380 quotes a second. [This last description was clarified thanks to the kindness of author Joe Flood.]