Further Fun with Forecasting
The trend is your friend
In my last post, I started a practical evaluation of the accuracy of the binary markets on Manifold by betting against the market (ie. towards 50% probability) on every market I could see. Based on some super-basic analysis of past markets, I predicted that we’d see the standard inverse-sawtooth barbell strategy pattern, where we would ‘bleed’ consistently, losing small amounts of money on most bets but making the occasional massive win. Backtesting against historical trades predicted profits for 2,000 trades reaching anything from M$33k - M$33mn, depending on how you excluded markets. However:
[The risk is] bleeding to death long before your strategy pays off - and that’s a problem that’s likely to get worse if Manifold’s markets get more accurate and less prone to obvious silliness and manipulation.
So, three months later, how has this worked out?
At first glance, this appears to both vindicate the strategy and my prediction. Unfortunately, the reality doesn’t match the graph.
Only the last of the upward spikes on the chart actually came from the strategy itself; the others were from an out-of-strategy capitalization on Boris Johnson’s not-a-resignation and from signup bonuses from that market. While the portfolio balance still showed as positive, that was largely due to a large number of open bets running at various degrees of profitability; by August I was down to my last few M$.
In mid-August I decided, therefore, to switch strategy and test the inverse; instead of betting against the market, I would bet with the market as it approached resolution. I forced myself to be as dumb as possible in applying this; I ignored all my own instincts on almost every question and, apart from filtering out a few troll questions & posters, applied almost no judgement at all. I wanted to simulate as best as I could the outcome of someone with blind trust in the accuracy of prediction markets.
The results are pretty good:
Note the smooth, dull, leisurely climb to profitability.1 Since reversing the strategy, I have bet on 429 markets, ~80 of which resolved against me - there being some overlap in August as bets made under the previous strategy resolved while I started the new strategy.
Interestingly, this is also in line with the backtested strategy of the previous post, which suggested that we’d triple our money in ~2000 bets. Reversed stupidity is not intelligence, but it also doesn’t make sense that two completely contradictory strategies could both be in the money - so what’s going on?
I think the most obvious reason that the counter-market strategy failed is that I had overestimated the profits I would make from successful contrarian bets. In my crude backtesting, I’d assumed that I would always be able to buy any number of shares at the stated price. In reality, however, there are only finite shares available and not all of them are available at the current market price, which severely limits the return on investment of a contrarian bet. In other words, I hadn’t accounted for slippage. It’s still possible to turn a M$10 bet into M$100,000, but very, very few markets will be both totally mispriced and have enough liquidity available for someone to take massive profits from that mistake.
This in turn suggests that we may eventually make a serious profit by targeting only those markets which have high liquidity and are heavily skewed to one result. As BionicDolphin pointed out in the comments of the last post, using limit orders might work well here as an execution tool. If I can find the time to put a bot together, I think this is worth pursuing - but for my day-to-day activity, I’m happy to keep betting with the trend, which, in turn, is a ringing endorsement of prediction markets as a forecasting tool. Well done, Manifold.
Bootnote
You may be wondering, given the apparent success of this strategy, why my profit is showing as negative in the charts above. The reason is simple: I am an idiot.
So far we’ve been looking at my portfolio (my cash balance + current value of all open bets). Let’s look at my profit instead (ie. cash extracted from successful bets only):
The inverse saw-tooth is still visible in July; the dip into negative profitability in September comes as the last of my counter-market bets resolve themselves, and then bounce into profit on the death of Her Majesty. The slow climb upwards vindicates the pro-market strategy, and then…
Yes, in my haste to profit on yet more royal news before the Americans woke up and closed the market, while trying to put my children to bed, I somehow managed to mix two dates up in my head and bet my entire balance the wrong way. By the time I realised my mistake (thank you Heaffey), it was too late.
There’s not much to say about it. I ran support for thousands of traders over five years; I've heard too many pannicked traders on the other end of the phone asking if there were some way for their trade to be reversed; I know that mistrades happen. Sometimes, in ‘real’ markets you can get the broker or the counterparty on the phone and get the trade reversed (depending on how big and important you are and how much the broker depends on your continued custom, this can be anything from trivial to impossible); sometimes you can pay an exchange some significant portion of the trade size to back out of it. But in the wonderful world of Web3 and techy-financial enterprises where Code is Law, you’re stuffed.2 And in this case, I deserve to be. My greatest regret is that I don't have a funny story to excuse it, unlike the broker who once told me that the €100,000,000 trade that just blew up our screens had been entered by his cat rolling over on the keyboard.
The little step in September was a break in strategy in which I correctly predicted the Queen’s death on multiple markets when I heard that the family was being called to Balmoral, and then felt a bit grubby about making money that way and donated it.
To the great credit of the Manifold team, with whom by complete coincidence I happened to be chatting not long after this loss, they did immediately offer to reset my balance. I declined.