Rise of the Algos – Knight Capital and the Inevitable Catastrophic HFT Market Meltdown

By mw | Related entries in Bad Decisions, Banks, News, Regulations, Wall Street, WTF?
We're the HFT Algos and we're here to help provide liquidity.
We’re the HFT Algos and we’re here to help provide liquidity.

It happened again on Wednesday.  A High Frequency Trading  (HFT) proprietary algorithm (algo) crafted by highly compensated brains-on-sticks and nurtured in the bowels of an investment bank escaped into the wild. From its carefully co-located massively parallel high speed trading computer it emerged to wreak havoc on the financial markets.  In this case, it was a happy ending. The algo turned on its master and sucked the very life blood out of the firm that deployed it.

The incident is yet another lights-flashing, klaxon-blaring warning signal that competing HFT algorithms operating with the speed, size and complexity that are commonplace today cannot be adequately regulated, tested, or managed. Their interactions with each other cannot even be fully understood or predicted.

Wednesday 8-1-12: The Knight Capital Algo Clusterfrack

John Carney at CNBC:  Latest Market Glitch Shows ‘Trading Out of Control’

“Wednesday morning’s stock snafu had a familiar ring to it — mysterious volume in trades that simply could not have been made by a human comes surging out of nowhere, causing brief but acute market mayhem By now, many players on trading floors have gotten used to the disruptions that can come from the highly automated new world of high-frequency trading. But that doesn’t mean they like it. “This algorithmic trading is kind of out of control,” Phil Silverman, managing partner at Kingsview Capital, said as officials at the New York Stock Exchange tried to make sense of what happened… Authorities involved in reviewing the matter said Knight Capital, a trading outfit that employs algorithms used in high-frequency trading, said it experienced “technology issues” with its market-making procedures.”

Once Knight Capital’s algo was identifed as the culprit, the public company’s stock was slashed by 1/3 – a $300 million market cap haircut decapitation. That was Wednesday before we knew how much the algo lost on the trades. On Thursday morning KCG’s stock was slashed in half again when the trading loss was being reported at $440 Million.

Tyler Durden at Zero Hedge: This Is What Happens When An HFT Algo Goes Totally Berserk And Serves Knight Capital With The Bill

“We also all know that one should buy low and sell high. At least that is what human traders are taught, and that is what they attempt. Because if one consistently does the opposite, one will simply run out of money. Well, the opposite is precisely what the berserk algo in Knight’s Market Making group may have done if Nanex, which has done a forensic analysis of one of the trades in question, is correct. In other words, instead of at least attempting to provide liquidity via limit trades, Knight’s algorithm acted as a market order… gone horribly wrong. As the third chart below shows what the algo did with furious repetition and steadfast consistency was to buy at the offer, and sell at the bid, in other words buy high and sell low. Over and over and over and over. As Nanex laconically notes, “In the case of EXC, that means losing about 15 cents on every pair of trades. Do that 40 times a second, 2400 times a minute, and you now have a system that’s very efficient at burning money.”

The algorithm ran 15 minutes. It lost $440 Million.

 HFT SNAFUS Are Becoming Commonplace With Increasing Financial Damage

Wednesday’s event provided an opportunity for investment professionals, financial news networks, and market pundits to reprise previous HFT failures.  Once again they can wag their collective  fingers about the complexity, lack of safeguards and unknown dangers of these high speed transactional engines.

Paul Vigna at Wall Street Journal: When Robots Strike: From the Flash Crash to Today’s Snafu

“Today’s trading irregularities, or error, or snafu, or whatever cutesy term this one ends up getting tagged with, is not the first mistake of its kind in the computerized, high-frequency world of the stock market. The 2010 flash crash is the most notorious example, but it isn’t the only one, and indeed, these kinds of errors seem like they may just become an irritating but regular fact of life for traders and investors. Here are some of the most notorious examples of computer-driven snafus. Welcome to the new stock market… Flash Crash…. BATS IPO… Facebook IPO… Sawtooth Trade….”

This is the problem. These “glitches”, “SNAFUs”, “hiccups”, “errors” and “oops”  are happening with sufficient frequency that they are now being treated as unfortunate but normal aspects of today’s markets. They are not.

These failures are warning signs that the high frequency nuclear fire burning in the furnace of the financial reactor is spinning out of control,  the containment  has been breached,  and unless the transactional chain reaction is cooled or shut down a catastrophic meltdown is inevitable. But apparently nobody wants to believe that. Instead we see the regulators and executive leadership on Wall Street tapping on the flashing red indicators in the financial control rooms and saying “Do you think this thing is really working?”

Criminal Algos Are a Big Problem But Not The Biggest Problem

Forget that competing algos executing hundreds of thousands of high speed financial transactions every second are indistinguishable from the illegal practice of Quote Stuffing and Front Running. Forget that if the identical transaction patterns were being placed by individuals at speeds that humans could comprehend we’d be seeing perp walks, indictments and convictions. None of this is new. Justin was blogging about HFT in 2009. It is well known that no one is enforcing the law when algos are committing the crime. Is this unconscionable malfeasance on the part of  private and public regulators and compliance officers? Yes. Nevertheless, the consequences of quasi-criminal HFT trading amounts to mouse nuts when compared to the potential financial damage that these algos could and probably will ultimately inflict on the market. The bigger problem lives at the intersection / interaction of competing HFT programs and trading systems.

“Price is truth” is a popular aphorism among traders on Wall Street. These algos are designed to secure a pricing advantage over other algos and human traders by setting, quoting, offering, cancelling and transacting price quotes. It does not matter how much testing on each individual algorithm or program is done within the vacuum of a market simulation.  It is inherently impossible to predict how the algo will behave in every circumstance when competing with many other algos that are gaming the system for the same price advantage.  And it all happens so fast and and at such high volume, that by the time humans can intervene and pull the plug, the damage is done.

Undoubtedly there is a PHD dissertation and possibly a Nobel Prize to be earned by a Chaos Theory scholar proving that:  1) Complex dueling pricing algorithms will always yield unpredictable results  and 2)  It is mathematically impossible to eliminate the possibility of these systems unintentionally precipitating a market destroying “black swan” event.

A Slow Motion Example of Dueling Pricing Algorithms Gone Awry

An appreciation of the kind of unintended consequences that can result when pricing algorithms compete can be seen in a much simpler example that occurred over a much slower period of time. Consider The Case of the $23.6 Million Dollar Book On Amazon:

“…how about this: $1.7 to $2.1 million for a book—a perfectly average, out of print but not quite rare title that’s considered a fundamental work of development biology? Or how about… $23.6 million?  That’s the result of a recent bidding war between two third-party Amazon merchants, each attempting to use algorithms to sell an out-of-print version of Peter A. Lawrence’s, “The Making of a Fly: The Genetics of Animal Design.”  UC Berkeley associate professor Michael Eisen first noticed the pricing irregularities when he attempted to pick up a copy of the title a few weeks ago on Amazon. The book–with a list price of $70 on Amazon–was only available as “New” from two third-party merchants: “profnath,” who was listing the title for $1.7 million, and “bordeebook,” who one-upped his competition with a price of nearly $2.2 million.”

Of course compared to the HFT algorithms, this is a stupidly simple, wildly unsophisticated and incredibly slow example of dueling price algos. The point is that neither individual algo was a problem in and of itself. It was the interaction of dueling algos that created a preposterous price by making dozens of price quote adjustments without human oversight over a period of weeks. Now – Is this at all applicable to the myriad of  proprietary complex pricing algorithms independently competing and seeking to game the market by adjusting prices at a rate that precludes human oversight?  Hell yes.

Which is the Dog? Which is the Tail?

The question is begged: Even if there are feral pricing algorithms running wild in the trading marketplace, operating at extreme high speeds, and partaking in questionable activities – can they really be a danger to the ginormous equity and commodity market?  After all, they are mostly trading in very small increments and  arbitraging fractional price differences across stock exchanges. How big an effect can that really have? Is this not a tempest in a teapot?  Tyler Durdan at ZeroHedge offers this remarkable Nanex animation graphically presenting the impact of HFT activity on every major exchange since Reg NMS made HFT possible.

The Rise of the HFT Machines – by Nanex

This looping animation shows the daily packet traffic (approximately translates to transactions, quotes, orders, and cancelled orders) at each of the US market exchanges from January 2007 to January 2012.  This covers the period time when HFT became an integral part of Wall Street. You can watch the date scrolling in the lower left corner. The biggest flareup in this animation occurs in August 2011 when US credit is downgraded to AA+. The HFT quants made a lot of money then.  It is certainly fair to ask – Who is really wagging what?

“Kill the Quants Before They Kill Us”

An oft repeated refrain from talking heads on the financial news networks is that we need more human oversight in the pricing process to provide a  common sense circuit breaker for HFT algos.  It was the traders on the floor of the New York Stock Exchange that called attention to the berserk Knight Capital algo and the plug was pulled within 15 minutes of it’s assault. In those 15 minutes Knight Capital took $440 Million in trading losses, exceeding their entire cash holdings. In the 36 hours since, Knight Capital has lost 80% and  $800 Million dollars of market capitalization. It seems likely the company will not exist in it’s current form by Monday.

The only thing I can think of that is remotely comparable to this notion of human oversight and management of massive number of hyperfast events is the control room of a nuclear reactor.  There human operators monitor data on dials and screens that are hopelessly behind what is actually taking place in the nuclear furnace. The conceit is that engineers can design systems and algorithms that will anticipate every possible thing that can go wrong and these tools will give the human controllers time to react. Most of the time it works. But when it goes bad, it goes really, really bad. The engineers simply did not anticipate everything that could go wrong at Three Mile Island, Chernobyl, or Fukishima.

I am with Doug Kass on this one:

“I say kill the quants and their technology before they kill us,” he exclaims on CNBC’s Fast Money Halftime Report. Kass, who is president of Seabreeze Partners and a CNBC Contributor made the comments with a wink in his eye.  But he’s very serious about events involving Knight Capital and the ripple they will likely send through the markets. Though relatively small – Kass worries they may be the proverbial straw that breaks the camel’s back.“It’s just the latest gaffe in a series nightmares on Wall Street – it’s as if Freddy Krueger is taking and axe to the heart of investors.” Kass points to other technology related woes, such as the Facebook IPO debacle in May, the glitches on BATS in March and of course the Flash Crash on May 6, 2010.”

We’ve had plenty of warning. The lesson of the Knight Capital “glitch” is unambiguous. You can test individual algos all you want but you cannot anticipate every consequence of what may transpire when dueling algos independently fight extended wars for profit within the blink of an eye.

Unfortunately it is not in our collective psyche to react to warnings.  Whether it is the threat of terrorism and 9/11 or the impenetrable synthetic derivatives bundling sub-prime mortgages and the subsequent crash they precipitated – we had plenty of warnings. But we ignored them. We only seem to fix problems after a disaster and a lot of people are hurt.

I guess I’ll be linking back to this post shortly after the rogue HFT algo disaster in our collective future finally comes to pass.

See you then.

X-posted from The Dividist Papers


This entry was posted on Friday, August 3rd, 2012 and is filed under Bad Decisions, Banks, News, Regulations, Wall Street, WTF?. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

14 Responses to “Rise of the Algos – Knight Capital and the Inevitable Catastrophic HFT Market Meltdown”

  1. Jim S Says:

    Wow, mw and I can agree on something.

  2. mw Says:

    Wait. What? Who are you, and what did you do with Jim?

  3. Jim S Says:

    Yep, mw, algorithms that do nothing but attempt to track the flow of funds in order to buy and sell at the proper millisecond are, IMO, extremely harmful to the markets and the economy. First, it does not accomplish one of the primary things that a stock market should do, which is allow the stock price to reflect the actual value because of underlying fundamentals of a company. That’s hard enough to accomplish in which the main goal for too many organizations is to produce churn for the sake of transaction fees but when the actual majority of transactions in a market consist of those being done by algorithms then it’s just this side of impossible. And while it’s hard to tell with certainty, I’ve read that some analysts think that we’ve reached that point now.

    In addition, if everything you do depends on speed then at some point a manager will expect the same from their programmers. Fast coding does not necessarily (and in fact rarely) produce good code. And to give you an idea of how much speed matters to these businesses, many of them make a point of colocating their servers in the same server farms used by the NYSE and NASDAQ so that the speed of transmission lag won’t hurt them. And one thing that I wonder about is the interaction between “competing” algorithms in the wild. What kind of interaction might happen with so many transactions being executed at such speed with different programs with the same general intent but differing methodologies? I don’t think anyone has a clue, including the folks writing and using these programs. The possibility for disaster is great, IMO.

  4. mw Says:

    “What kind of interaction might happen with so many transactions being executed at such speed with different programs with the same general intent but differing methodologies? I don’t think anyone has a clue, including the folks writing and using these programs. The possibility for disaster is great, IMO.” – js

    Exactly. At these speeds and volumes, the only way disasters can be averted is if the programmers can anticipate every single thing that can go wrong in these algo interactions before they happen and build those safeguards into the software. The notion is absurd on its face. The interaction of competing trading algos that are all being updated independently to work at cross-purposes with other algos is a more complex interaction with more variables in play than controlling a nuclear reactor. And they still can’t anticipate everything that can go wrong with one of those.

    This may be exactly what happened to the Knight Capital algo. Their program did not change, but one of the systems it interacted with did change:

    What exactly caused this “technology issue”? Knight still hasn’t told us. I mentioned several times yesterday that the NYSE had launched a new program Wednesday, the Retail Liquidity Program (RLP), essentially a dark pool inside the NYSE. It is quite possible that Knight had a problem interacting with that program due to some kind of coding issue. Other firms apparently interacted with the RLP without incident.

    And BTW, Exactly why do we now need a “dark pool” inside the NYSE? I just don’t get it. You’ve got big time really smart investors, movers, shakers, analysts and technologists all saying there is an iceberg dead ahead, yet the only thing that appears to be happening is the formation of deck chair reorganization study committees.

    This makes no sense.

  5. Jim S Says:

    The dark pool is a very, very bad idea whose only purpose is to make money for traders using highly questionable and extremely risky methods, IMO. It’s like the shadow economy in CDS’s that was worth trillions of dollars that contributed to how the crash played out across the world markets. The excuse that all of these folks use is the necessity for additional liquidity in the markets. But I think it’s highly questionable whether or not high speed transactions or dark pools actually contribute to market wide liquidity in a meaningful fashion.

  6. Tully Says:

    When you build volatility into the system AND everyone develops their own ways to exploit it, you greatly magnify the potential for chaotic results. And I mean “chaotic” in not just the normal sense but in the mathematical and societal senses as well.

    Since none of us (that I know of) are program-trading investment bankers poised to profit from such chaos, I think it’s safe to say we all have a common interest in the discovery of ways and means to prevent that. Some have proposed trading transaction taxes, which could reduce the mciro-second trading somewhat. To me that appears to be motivated just as much by potential tax revenues as actual efficacy. My own preference would fall along the lines of a random variable time buffer on trades. Tough to “ace” and micro-arbitrage when you don’t know if it will take a half-second or ten seconds for your trade to lock in. The increased uncertainty would in and of itself reduce the potential for runaway chaotic effects.

    Yes, folks, there is a huge area of common ground. Now if only we could get our pols to quit insisting we must march in goose-step with their pseudo-religious party dogma and doctrine. They rule by keeping us divided with wedge issues, but on MOST issues there really is no difference other than preferred approaches to reaching the already-known desired solution.

  7. mw Says:

    I don’t know if this more or less fun with all this agreement. I find it… unsettling.

    The guys at NANEX have done extraordinary work showing exactly how these HFT debacles happen. They created the animated gif in my post showing the massive increase in HFT transactions over the last two years. In this post they document exactly what the rogue Knight Capital algo was doing within a single second of trading. Apparently Knight deployed a new trading algo, and also deployed the market simulating algo that it was being tested against -and both began running in the real market. Within minutes $440 million was lost. I just cannot get over it.

    I’ve been watching the HFT defenders over the last couple of days. They basically want to present HFT risk as the same as any other new risk in the market. It’s the new normal, a technological advance, just a new part of the everyday investment environment that must be addressed with the same sort of risk mitigation policies and oversight that every investment bank must deal with every day. When mistakes are made, failures result, and Knight capital is just one more example. They paid the price… nobody else was affected… blah… blah.. blah.

    I think this notion can be rebutted with a reductio ad absurdum. We know via Moore’s laws and computing history that computers will only continue to get faster and more powerful. The end game is quantum computing:

    “Technology changes and moves faster than most of us realize. The processing power that many companies have harnessed is faster than most ever thought possible, but as fast as our current computer technology is, it remains quite slow. The world’s fastest super computer, Japan’s K computer has approximately the processing power of one human brain. There are some things the K computer can do as fast as an average human, some things such as pattern recognition, more slowly. By comparison, and although we are still very far from mastering this application of quantum mechanics; researchers have estimated that a quantum computer no bigger than a laptop has the potential to perform the equivalent of all human thought since the dawn of our species in a tiny fraction of a second!”

    Whether we ever get to that level or not, consider the consequences if the only constraint in HFT is the limit of the technology. It is certain that within a few years or decades the total number of HFT transactions that were required to lose $440 million over 30 minutes or so for Knight Capital will be able to be executed in 30 milliseconds. Who is going to pull the plug on that mistake? Instead of taking 30 minutes to destroy a company, a company could conceivably be destroyed in less than a second after deploying a flawed algorithm. What happens when a company with $300 million in cash loses – not $440 million – but $10 Billion? $1 Trillion? What happens to the market then? How does that get sorted out? It sounds absurd, but the Knight Capital FU informs us it is certainly possible. And if it is possible, despite all human efforts to prevent it, we know Murphy will eventually make it so.

  8. cranky critter Says:

    Yeah, we’d be fools to think that this one delightful outcome is the one that will be replicated. Instead, we likely end up suffering bizarro volatility and even weird “mathematically chaotic” trends where all but the super literate math geniuses get totally creamed. Eff that.

    As Tully implies, the right question is “Is anyone else besides high frequency traders benefitting from high frequency trading? If we think the answer is pretty much no (and I sure do), then eff those traders, plain and simple. Randomizing transaction speed would almost certainly be a sufficient wrench into the works.

  9. mdgeorge Says:

    I hate it when captchas eat my posts.

    Anyway, in my experience with statistical algorithms, adding random jitter doesn’t help – because these things happen with high frequency, the jitter will always average out to zero. The principal is a little different here, but I’m pretty sure that the people who do this stuff would just adapt to the jitter and continue with AHFT – almost high frequency trading. Randomization would reduce the benefit, but not eliminate it.

    If there continues to be effectively zero cost, then any benefit, even a small one, will be exploited.

    If the delays were big enough to put the algorithms on the same time scale as the human traders, that might level things out a bit – say that the delay was on the order of minutes. But if you were to go that route, you’d probably just avoid randomization and put everyone on a 5 minute schedule.

    I think the penny per transaction method is simpler and more likely to work, in the same way that I think a penny-per-email scheme would be the best way to end spam. It’s even better here because the systems are already and centralized (unlike email), and because the added revenue would be a welcome thing. But as you can see, revenue is not my primary motivation for advocating that scheme.

  10. Tully Says:

    The “super literate math geniuses” would get creamed too. Pride goeth, comeuppance, etc. And I’m good with that. It’s everyone else getting creamed because of the hubristic manipulations of the “super literate math geniuses” I am against.

  11. Angela Says:

    These math geniuses may be “super literate” but they lack common sense. If X then Y, if Z then Q, they haven’t perfected the algorithms, obviously. Again, I’ve said it before, its using technology just because its there.

  12. Angela Says:

    Its not just the speed at which trading occurs, the problem lies in the algorithms as well.

  13. mdgeorge Says:

    The real problem is that investment is completely decoupled from understanding what businesses are doing, and evaluating their utility, and instead is based on trying to extract money from noise (otherwise known as gambling).

  14. Angela Says:

    Those are my thoughts mdgeorge, exactly. How do you solve for that mathematically?

Leave a Reply


NOTE TO COMMENTERS:


You must ALWAYS fill in the two word CAPTCHA below to submit a comment. And if this is your first time commenting on Donklephant, it will be held in a moderation queue for approval. Please don't resubmit the same comment a couple times. We'll get around to moderating it soon enough.


Also, sometimes even if you've commented before, it may still get placed in a moderation queue and/or sent to the spam folder. If it's just in moderation queue, it'll be published, but it may be deleted if it lands in the spam folder. My apologies if this happens but there are some keywords that push it into the spam folder.


One last note, we will not tolerate comments that disparage people based on age, sex, handicap, race, color, sexual orientation, national origin or ancestry. We reserve the right to delete these comments and ban the people who make them from ever commenting here again.


Thanks for understanding and have a pleasurable commenting experience.


Related Posts: