Real money, invested with algorithms(blog.quantopian.com) |
Real money, invested with algorithms(blog.quantopian.com) |
Trading is hard. I run a proprietary trading firm with modern technology infrastructure and we have a lot of trading experience. You really have very little chance of success unless you're doing this full time, and even then it will take years before you have enough experience to build a sustainable income.
If you look at a typical discussion of trading here on HN you will see lots of totally uninformed hyperbolic assertions about what goes on in the market. How everything is rigged and how corrupt the players are. I think trying to scare people out of the markets only promotes that kind of thinking.
IMO there's way too much fearmongering (largely from people who have no experience trading, or from traders who don't want to see other people do better than them) about this. I've traded options on a very volatile commodity ETF before (SLV) and came out quiet well on top by playing with patterns and timing. This was in 2010, but there was the same "THE COMPUTERS CONTROL THE MARKETS AND YOU'LL GET CRUSHED" fearmongering hyperbole as there is today. It'll be there four years from now, too.
There's smart money and super fast algorithms and all that for sure, but there's plenty of dumb money moving around too. There are also plenty of people (I will note: often nontechnical individuals) who make a good living day trading. You basically just have to find a pattern ANYWHERE, IN ANYTHING and trade it over and over until it stops working (...and catch on quick). You might lose money (especially starting out), but it's not at all the experience of "you will step in and BE CRUSHED BY EXPERT TRADING ALGORITHMS" that people portray it as.
Start a brokerage account, do some reading, put in a sum of money that's consequential but that won't break you if you 100% lose it, and play around a little bit. Try focusing on one specific (preferably fairly liquid) security. You'll be amazed at how sometimes you make the right call and magically you have more money than you started with, and how you don't immediately die and get shot and lose all of your money to some corrupt evil high-speed algorithm while being eaten by wolves because, you shameful Icarus fool, you dared to trade on the public markets by yourself.
Risk-averse people (especially programmers) who have no experience trading will think you're completely insane and lambaste you, especially if you make money.
Yes, one does see such remarks, but an excellent counterargument is almost never heard-- and that is that (a) business leaders aren't stupid and (b) if the equities market was really significantly rigged or inefficient, these people would refuse to raise capital using equities.
The fact that some of the most skeptical, cautious people in existence, people with plenty of practical experience, are willing to fund their businesses with equity capital, means that equity trading is substantially fair and that the efficient market hypothesis is at least approximately true.
Heh. I also wanted to play with algorithmic trading, but before doing so, I got a book "Trading and exchanges" by Harris, which was recommended by somebody on reddit.
I have read it only halfways and realized two things:
1. I'd have to do trading (and related activities) as a full-time job in order to be successful (i.e., earn money in the long run).
2. I'd need to spend at least a year's equivalent of a full-time job on learning the necessary background math and algorithms.
I.e., I'd need to quit my job (which I quite like), only to replace it with another set of daily duties.
If it were easy, and everyone could do it and make as much money as working full-time, without actually working full-time; then everyone would.
That's easy to answer -- if you're very smart, you will do better in math and physics, and the positive correlation between intelligence and success actually has meaning. But in equities trading, being very smart does no good at all, and might even represent a handicap. The reason? Equity pricing is mostly random noise, very high entropy.
Many very smart people have tried to model and then beat the market, and (apart from chance outcomes) no one has succeeded in the sense that they located a reliable, describable "method" for it.
Also, if in principle someone discovered an actual winning system, by publishing the system he would destroy its effectiveness (because once a "system" is put into practice, once everyone uses it, it loses its effectiveness).
Obviously for a random collection of investment schemes, half will do better, and half worse, than the average market, but the half that do better will naturally enough claim the outcome resulted from their investment genius -- even if they're smart enough that they should know better.
I can use a computer to model a random market and random investors, using random price changes and random trades, and not surprisingly, half the investors will do better than the average outcome. What is surprising is that, when the above-average investors are humans, they invariably try to claim this chance outcome resulted from a winning system that they will sell to you for some princely sum.
More here: http://arachnoid.com/equities_myths
> Pointing that out X is hard and most people fail is a tautology, it's true of all difficult professions.
Yes, but there's a qualitative difference between "hard" and "impossible".
I wouldn't say no one should trade -- obviously some people do very well. But most of us, I think, should be very careful, and limit our exposure.
This kind of strategy enables laymen like myself to do about as well as the market, without paying a middleman such as a mutual fund.
The answer is that buy & hold is a very reliable way to grow your capital, and it does without the services of investment advisors and brokers, which is why they don't want you to find out about it.
On average, buy & hold investors do better than those who enlist the help of brokers or who become day traders (who on average do worse than anyone else).
For actual evidence, one need only examine the outcome of the Wall Street Journal dartboard contest, which compared expert equity picks to random picks. The expert picks weren't sufficiently better than random picks to justify the broker fees, and this outcome went on for years.
http://www.investorhome.com/darts.htm
Quote: "... the performance of the pros versus the Dow Jones Industrial Average was less impressive. The pros barely edged the DJIA by a margin of 51 to 49 contests. In other words, simply investing passively in the Dow, an investor would have beaten the picks of the pros in roughly half the contests (that is, without even considering transactions costs or taxes for taxable investors)."
But you're dead wrong if you think large-cap stocks provide safety from risk. Plenty of large-cap stocks have disappeared completely, another large cohort have had terrible returns.
Any strategy is only going to work if you put disciplined position sizing/money management rules in place. Rules that allow you to trim the dogs and keep the winners, whilst making sure that any large losses don't fatally wound your portfolio.
Of course there are -- very easy ways. But there's no reliable easy way to get rich.
> people looking to get rich quick in financial markets are probably better off learning how to play poker.
If we eliminate "quick", then not true. All one needs to do is invest in an equity index fund and patiently wait decades while ignoring the dubious claims of investment counselors. It's very reliable.
With respect to "quick", that also happens but with much less reliability. Consider that a blind computer program randomly flipping coins, with no strategy at all, will enrich a small percentage of its accounts by chance alone. This works for people too, but it's a rare, chance outcome, not any kind of strategy. More here:
Hm... actually, it might be. At the very least you certainly don't have to shell out as much for software.
We're trading on live data (not just making claims based on forward tests) without any human involvement, though human intervention and the ability to recognize trends could have saved some of the trades that ended up being losers. Still, we're looking at sizable gains with a hands-off approach (maybe you'll need to re-train the system every year, but even that can be automated).
We've now run into the problem of telling the world about it; you'd think that 40% annual returns would have large institutions knocking down the door to give us our millions. So it goes!
Trading on real data is very different from simulated trading on historic data. There are a lot of leaky abstractions and assumptions that happen during simulations that cause problems once you put real money to work. In the past, here on HN, I have criticized quantopian several times because no one has actually used quantopian to trade real money. This is no longer true.
It will take a lot of time and a lot more capital to figure out all the holes, but they are finally on the right track and I am glad they have 20 people already trading. That is 20 sets of eyes balls to look for lack of fidelity in the simulations.
Personally I am interested to see how good their transaction cost model is when trading hundreds to thousands of shares at a clip.
Personally, I couldn't wait for their roll-out. So I wrote my own IB client and traded my Quantopian backtested strategies myself on my IB test account. Race condition, deadlocks much at 3:59PM ET at 2000+/msg burst from quote ticks (esp. when you are doing some kind of stat arb, pairs trading where one order submission is dependent on another order's fill).
Anyways interesting technical problems abound. Gl to the Quantopian team!
does anyone know where I can get price history for the last few years for something like an index fund, like DIA?
To begin coding, clone the sample algorithm on the front page to get familiarized with the IDE. You can then backtest the strategy, for free, using historical data from 2002 - Now. When you run a full backtest, you can see how your algorithm would have performed in the past - you can see your returns, Sharpe ratio, and other metrics.
Once you're comfortable with the syntax and mechanisms, the community is a great place to look for trading ideas to try on your own. Try cloning an algorithm posted in a thread to begin experimenting. If you get stuck, post to the forums - they're a friendly bunch and will be happy to help out.
Of course, past performance is not indicative of future results. When you're ready, you can paper-trade your algo (trade using fake money) on the live market. It's free on Quantopian and a risk-free method to see how your algo performs in the market. And then, the last step is to connect your account to Interactive Brokers and begin live trading with real money. This blog post was an update of the live trading pilot program.
(Disclaimer: I work for Quantopian. But I'd be happy to help answer any questions! Feel free to reach out to us directly as well, feedback@quantopian.com)
Doesn't IB require a local TWS connection as well as credentials, e.g. 2-factor authentication?
EDIT: There are just too many exceptions and nuances. Every single investment involves more exceptions and nuances that would need to be coded. And if you don't think so, I would bet you don't have alpha :)
But... the solution is just so obvious, just make the algorithm trade the opposite of what it would have done.
Pure profit!
Here's the bot I built: https://github.com/pontifier/rebalancer
Quantopian looks interesting, and hopefully the exchanges it trades on won't disappear.
on Techzing podcast can find and interview to the founder (a bit old, 2011): http://techzinglive.com/page/868/155-tz-interview-mathew-kle...
Feel free to upvote this comment if you think this is cool btw :)
Fore more info see: https://www.youtube.com/watch?v=hye1L86s43M
Reality is that if I have $10'000 to trade, unless I am successful, which is likely a low probability; 10% of that goes to brokers for all the spreads I pay over time, and the other 90% simply goes to another man/woman/company's pockets.
The one thing that I discounted was the adrenaline rush in seeing a certain currency pair or index go up by a few bips in a matter of milliseconds and that translating into gains/losses in my account. If you do it to 'try' it out, stay away. If you don't understand the effect of a certain EMA converging/whatevering with another EMA, stay out. If Bollinger rings of a friend's surname more than a band, stay out. etc. To even be extreme, I would equate high-leverage trading to gambling for us folk who don't have a lot of knowledge, and I'd go further and equate it to drugs for those who have less of a clue.
As the parent post said, stick to value investing.
Disclaimer: I've spent about 6 years trading in my spare time, learnt lots etc. but called it quits 3 years back to focus on something better with my time. I'm currently 10e10/4-.5 years old.
http://www.jatit.org/volumes/Vol51No1/24Vol51No1.pdf
Also, weren't the quant funds that were involved in the summer 2007 blowup all running automated value strategies based on Fama+French factor models?
It works really great until it doesn't.
It depends on how you define "works". A random strategy doesn't work at all on average, it can only be said to "work" for the 1/2 of the investors that by chance fall above the average.
And a random sequence is by definition a series of black swans. Unless you're saying that a black swan can only be an extraordinarily unlikely occurrence, in which case all we need do is wait a little longer for the improbable but eventually certain outcome that will meet anyone's definition of black swan.
There are 23 people in a room talking. Pretty quickly two of them realize they have the same birthday. Is that a black swan? After all, there are 365 1/4 days in a year -- how unlikely is it for two of 23 random people to have the same birthday? Is it 23 / 365.25 (6.2%) or is it more or less likely?
My point? Not all apparent black swans are really black swans.
To your first point about our development strategy: we used genetic programming, and tried to apply novel strategies for which there wasn't much of a precedent in financial applications.
To your second point, re: our testing strategy:
I will concede that we only have 2 months of live trading data, but so far it corresponds well to the behavior we were seeing in our forward tests (what we call the data that the system was not trained on).
We trained our system on 10 years of data (2001 - 2011) with two years of forward tests (2012 and 2013) for evaluating its effectiveness (and to verify that our algorithms have not been overtrained to fit the testing data set). Our forward testing involved taking the same data we would have received from our financial institution and feeding it through the algorithm to see what trades it makes. Having proven our algorithm worked with forward tests, we hooked it up to live data to get real results. So far, so good!
To you third point--risk--we need to explain how our algorithm trades.
We've trained our system to recognize patterns such that it will open a trade with a stop-loss and a take-profit. Because the take-profit and stop-loss are preconfigured, our system doesn't need to monitor data feeds with a time resolution of “real time” to make decisions; the onus is on the brokerage to close winning and losing trades.
The lot size of the trade is calculated such that a losing trade (the price of the currency pair reaching the stop-loss) will incur a loss equal to the risk. So at a risk of 2%, a losing trade on a $1000 account will bring the account value down to $980. Winning trades earn double the risk, so a winning trade on an account of $1000 at 2% risk will increase the account value to $1040.
With all this in mind, we claim 40% annual profit because our live trading exhibits patterns similar to our forward tests, and while our forward tests exceed 40% annual profit, we're extrapolating our two months' worth of live testing performance out to the rest of 2014. There's a good chance that this number will change in the coming months. We will revise it up or down accordingly.
It's possible that our system will need to be used "in the wild" for more than just two months in order for people to take us seriously. If that's the case, I am happy to continue dogfooding FX Machine; so far, it’s been a profitable experience :)
Thanks again for your great questions. We’ll use questions like these to build an FAQ section on the site!
I am not purporting to claim that FX Machine will always return 40%+ annual profit forever. Rather, I'm presenting a technology we've developed (as a startup on a startup site!) that trades automatically and, so far, has yielded returns that average (across all currency pairs) in excess of 40% annually. This number was higher on our forward tests, but we have two years of historical data for those and only two months' worth of live testing.
As I said in my reply to ep25, there’s a good chance that our 40% claim will need to be revised either up or down as we continue to dogfood our product. Investing is never a sure bet, even (or especially) when computers are involved. For that reason, I will continue to invest only money that I already have. I'm dogfooding our product with my own hard-earned cash--not somebody else's. What greater measure of confidence is there than this?
The question is not about 40% or 100%, if you are beating the market by a good margin consistently, why would you share that secret with everyone else. You trade/invest for your own account and get rich.
If your system cannot beat the market consistently why bother then, just close your shop and invest in an index fund.
If, however, we find an institution who understands the potential of FX Machine and is willing to invest in it, we can amass our millions much sooner than we would on our own. The opportunity cost for us associated with selling FX Machine versus using it personally is not measured in dollars; it's measured in years.
Are you asking why people invest in equities? It's because the market as a whole grows over time, sometimes as high as 15% but at present about 4% after taxes, fees and inflation. This means a buy & hold investment returns a reliable long-term averaged 4%, at a time when virtually no other investment can offer that kind of return (when expressed in the same way).
The above is true if one simply invests in an index fund that tracks (for example) the DJIA, and no trades are made -- no brokerage fees, no taxes until a redemption takes place, and lower taxes than for regular income.
Or are you asking why people have managed portfolios? It's because of widespread ignorance and a bit of the cowboy mentality -- a real cowboy would throw the dice instead of playing it safe. But "throwing the dice" in equities is easily shown to be a foolish act, historically, statistically, and reliably.
The average Joe does not know that.
I want to clarify that there's no real security in equities, my use of that term is only relative to other kinds of investments. Everyone should understand that you could lose everything, and there's no deposit insurance or anything like that.
What is the p-value that Warren Buffett's random investment plan outperforms the market 39/47 years?
http://finance.fortune.cnn.com/2012/02/25/buffett-berkshire-...
If a drug beat a placebo in 39/47 experiments, would it be approved by the FDA? :-)
There are investors who get lucky, and there are investors with sound strategies. The notion that every winning strategy gets immediately and flawlessly applied by every actor in the market is a fantasy world populated by homo economicus.
No, what I am saying is that random choices can produce the illusion of successful investment strategies for 1/2 the practitioners. This isn't at all controversial -- it's equivalent to saying that half of people are above average in intelligence.
> What is the p-value that Warren Buffett's random investment plan outperforms the market 39/47 years?
That's not a particularly interesting question (and it's not an improbable unbroken streak of 39 years, but a mixture of successes and failures, much more likely to result from chance). What is more interesting is to ask what Buffett's market performance would have been without the announcement and herd effects? Everyone wants to invest in the same stocks Buffett invests in, and Buffet's investments are public knowledge. The herd can be relied on to make Buffett's choices after he has established his position, which means he's positioned to benefit from the public's responses to his choices.
Again, I never claimed that investing is random choice. I only said that random choices work for 1/2 the investors, and those investors are likely to attribute their success to genius instead of chance.
Why is any particular permutation of 39 wins and 8 losses more improbable than another? That's like saying HHHTT is more likely than HTHTH.
> The herd can be relied on to make Buffett's choices after he has established his position, which means he's positioned to benefit from the public's responses to his choices.
But now you're playing it both ways. Are markets efficient or not? Can't every rational investor realize the herd will do this and just ape Buffett?
I may have misunderstood, but your thesis seems to be that markets are efficient, and no strategy can consistently beat the market over time. My thesis is that we have an existence proof that isn't the case.
Investing being random chance doesn't mean everyone will do about average. It means that some people will make quite a bit of money and some people will lose quite a bit of money. It just happens that Warren Buffet is an outlier.
We can say any return in strategy X is
1) Due to random chance (i.e., it defeated the market 39 times "randomly") 2) Not due to random chance, and there is an inherent advantage
I'm saying the probability of 1) is sufficiently low so as to believe 2).
That's an easy claim to make when you simply dismiss anyone who succeeds as random chance.
> by publishing the system he would destroy its effectiveness
Obviously and not disputed.
> I can use a computer to model a random market and random investors, using random price changes and random trades, and not surprisingly, half the investors will do better than the average outcome. What is surprising is that, when the above-average investors are humans, they invariably try to claim this chance outcome resulted from a winning system that they will sell to you for some princely sum.
That's not damning evidence, merely circumstantial.
> Yes, but there's a qualitative difference between "hard" and "impossible".
Yes there is, however I don't believe it's been adequately proven to be impossible.
I never said this anywhere. You're confusing the practice of science with a blanket statement I never made. Pretend to be a scientist -- prove me wrong. Locate where I said what you claim I said.
> Yes there is, however I don't believe it's been adequately proven to be impossible.
Oh, great -- "proven to be impossible". Before you go on, learn about the impossibility of proving a negative:
http://en.wikipedia.org/wiki/Russell's_teapot
Quote: "Russell's teapot, sometimes called the celestial teapot or cosmic teapot, is an analogy first coined by the philosopher Bertrand Russell (1872–1970) to illustrate that the philosophic burden of proof lies upon a person making scientifically unfalsifiable claims rather than shifting the burden of proof to others ..."
Remember -- all I have ever said is that the possibility of a chance outcome cannot, must not, be dismissed, and to a scientist, it is the first possibility to be considered, not the last.
> That's not damning evidence, merely circumstantial.
What? According to the scientific rules of evidence, the burden of evidence is not mine to prove that a winning stock picking strategy does not exist (i.e. prove a negative), the burden is on others to prove that it does.
Poorly worded, I wasn't claiming you said it, I was saying you could claim it and you would because you consider it the null hypothesis.
> Before you go on, learn about the impossibility of proving a negative:
I don't need to research Russell's teapot, I'm well aware, that was a poorly thought out statement on my part is all. I didn't mean to ask you to prove a negative.
> Remember -- all I have ever said is that the possibility of a chance outcome cannot, must not, be dismissed, and to a scientist, it is the first possibility to be considered, not the last.
It's not being dismissed. The issue is whether it's just chance; that Buffet's disciples have also done consistently well disputes the null hypothesis. Please actually read what Buffet has to say[1], he's not stupid and he's not ignoring the null hypothesis.
[1] http://www.businessinsider.com/warren-buffett-on-efficient-m...
That's a pretty bold claim. Given the financial crash of 08, from what I can tell the EMH has been thoroughly debunked. Markets are not efficient and all data is not priced in. Information is not absorbed quickly into the market so inefficiencies crop up everywhere all the time, those are the profit opportunities that traders make a living off of. If EMH were correct, the systematic risk that crashed the market could not have occurred. Remember, it's a hypothesis, not a theory.
Not at all. The EMH isn't falsified by people engaging in widespread cheating, and it isn't falsified by big market reversals driven by public psychology. It could only be falsified by the market's inability to accurately set a price on average, across all equities, perpetually.
Does an airline disaster contradict the claim that air travel is safe? No, that can only be contradicted by average flight outcomes. It's the same with the equities market.
> Markets are not efficient and all data is not priced in.
This claim is obviously contradicted by the fact that people are willing to use equities to raise business capital. If the market wasn't efficient, they would think of another way to raise capital -- something more efficient. That's hardly controversial.
> If EMH were correct, the systematic risk that crashed the market could not have occurred.
The EMH isn't falsified by cheating. Does the fact that insider trading takes place contradict a hypothesis that a market without cheating is fair and efficient?
> Remember, it's a hypothesis, not a theory.
Yes, and it can never be a theory in the scientific sense -- there's no way to gather objective data in a controlled way. It will probably remain a hypothesis in a pseudoscientific twilight zone forever. But given all the alternatives, the fact that people invest in the equities market argues for the truth of the EMH -- in an unscientific and dubious way.
In the end it's just an assumption in economic papers so they can be correct in some sense.
Apparently it can't be falsified or proved before the heat death of the universe, so not a very interesting or credible hypothesis.
This claim is obviously contradicted by the fact that people are willing to use equities to raise business capital.
jjarmusch, who is inexplicably hellbanned below as I post this, makes the point that people willing to use equities to raise money do not depend on an efficient market, in fact they profit from a broken or delusional market. So no, people using a market does not prove it is maximally efficient or even close.
The EMH isn't falsified by cheating.
No, it's falsified by comparisons with real markets which it attempts to model (which include cheating, stupidity and greed as well as occasional rational valuations).
However, relatively few people figured out how to efficiently make use of this information. Shorting the stocks of banks like AIG is very high risk, and can cost you everything if you don't know when the bubble is going to burst. In the end, the only shorting strategy that was effective was to use CDS, a newly popular derivative (which was a very smart idea). Jon Paulson's hedge fund made $15 billion from this, but Kyle Bass, another person who made money from this, says only 15 people figured out the CDS trade and made money from it. The amount of shorting that happened was tiny in proportion to the size of the bubble.
The EMH is a hypothesis that was contrived in an analysis that does not take account the actual way that information gets incorporated into prices, the actual trades available to market participants, or the asymmetry between upside and downside risks (aka. the Keynesian risks of opposing the market's animal spirits).
The EMH states that market prices correctly price in all public information. The weaker proposition, that it is hard to get rich quick from spotting market mispricing using just public information, does seem to be true.
EMH says that all publicly available data is priced in. The strongest form claims all data, but no one believes that (insider trading alone debunks it).
I would postulate that given the data available right before the crash, you would probably end up with similar results to what the pricing was at the time.
Note that one thing people have a hard time understanding with EMH is externalities, such as enthusiasm or whatever the opposite is called for a product or class of product.
> those are the profit opportunities that traders make a living off of
EMH relies on people making a living to function, the price setters need to do well or else enough won't be attracted to make the market efficient. You are correct that this kind of problem makes anything but the weakest form difficult to prove.
> the systematic risk that crashed the market could not have occurred
Why? One major factor in the crash was that people started walking away from their homes. Historically this didn't happen. If all of the data points to stability in mortgages, you cannot expect people to divine that problem, nor does the EMH require them too.
Insider trading doesn't "debunk" the EMH, because the EMH is predicated on fair dealing. There's a lot of distorted ideas about the EMH, for example that a market crash, or cheating, disproves it. These events don't disprove the hypothesis.
Wouldn't that depend on who the market is stacked for/against, by how much, and what their alternatives are?
For example, a business cannot possibly get one cent more than the purchasers of their stock offer in the marketplace, and because of transaction fees, somewhat less. And an investor has the same experience -- he can't possibly get more than the market value of his shares on the day of sale.
Brokerage houses are in competition to offer the lowest transaction costs and the most reliable environment for trading. That competition reduces transaction prices for everyone.
Conclusion? Not stacked in general. Insider trading is a crime, betting against your own customers is a crime, and these things happen, but they're prosecuted.
A rich person or big trading house can buy rack space close to New York or whatever city their market is based on, a regular person cannot. This ability gives a significant advantage to rich people/business owners, therefore the market is not fair at all.
People finance with VC money when there is not even usually a market.
People who want capital have little choice, they have to pay the ridiculous fees and have it traded in the existing market. But they don;t really care about the non competitive cartel of investment banks anyway, as running a public company means you can pay your self almost unlimited amounts of the shareholders money anyway. Whats not to like? Who cares if the market is efficient?
Next question. How many people need to be flipping coins for one of them to have a better than even chance to flip 20 heads in a row? Answer: about 3/4 million.
Next question. How many investors are there in the world? Answer: many more than 3/4 million.
Next question. If someone among millions of investors makes 20 successful market picks in an unbroken sequence, what's the probability that he will attribute that outcome to blind chance, and what is the probability he will start selling a book titled "Secrets of the Winners" on late night TV?
My point? When confronted by an unexplained occurrence, it's wise to consider the possibility that it's a random outcome. This is called the "null hypothesis" and it's the first possibility a scientist considers.
No, that's wrong. If the original performance had been an unbroken sequence of successful years, say, 39, it would be possible to apply the binomial theorem to it, but the binomial theorem would need to be applied both to the original probability and to the calculation of how many investors would be needed to create a better-than-even prospect of that outcome.
But the 39 successful years were randomly distributed among years where the performance wasn't better than market indices, which makes the probability much higher for it being explainable by chance -- how much higher depends on the actual pattern, which I wasn't able to find.
Then the hypothesis is useless and false. You can't claim markets are efficient and then exclude all evidence to the contrary.
> because the EMH is predicated on fair dealing
Yes, which makes EMH false in the real world. A hypothesis that's only true in an ideal world isn't useful nor is it meaningful in the real world.
You are not treating EMH like you should, think of it as a guideline not a hard and always true rule of life.
For instance lets say Telsa is trading for $100 and you think it is worth $50, there are two reasons for this, either:
* The market is not efficient and you should take advantage. * The market is efficient and you aren't considering all of the variables.
In almost every conceivable case it is the latter. The answer for Tesla is probably optimism for the technology when optimism often drives prices above the otherwise expected value.
EMH makes you consider the market as the baseline, which greatly helps steer you towards what is going on.
If that were true, if unprovable hypotheses had no practical value, psychology would collapse. Wait ... hold on ... nope, psychology isn't collapsing.
> So no, people using a market does not prove it is maximally efficient or even close.
You're missing the point that money flows to the most efficient of alternative capital raising methods -- which, if you think about it, also stands as evidence for the EMH. Given the freedom of businesses to choose any method to raise operating capital, and given that they prefer equities, this shows that the equities market is more efficient than existing alternatives.
If the equities market were less efficient than brand X, businesses would raise capital using brand X. How is that difficult to understand?
> No, it's falsified by comparisons with real markets which it attempts to model ...
You are apparently unaware that the EMH isn't compared to real markets, it's the other way around. And if businesses believed that equities were inefficient compared to anything else, they would change methods.
So in the best of all possible worlds, the EMH is true, but we don't live there.
> Why is any particular permutation of 39 wins and 8 losses more improbable than another? That's like saying HHHTT is more likely than HTHTH.
Again, that is not what I said. Are you trying to misinterpret, or is this all inadvertent? Had I anticipated your effort to misinterpret, I would have said that a random sequence of 39 heads is less probable than three sequences of 13 heads, each separated by some other unidentified, random sequences or, as I put it originally, "a mixture of successes and failures".
>> The herd can be relied on to make Buffett's choices after he has established his position, which means he's positioned to benefit from the public's responses to his choices.
> But now you're playing it both ways.
Nonsense. I'm explaining how any outcome can be attributed to chance, not that they are explained by chance.
> Are markets efficient or not?
That's not even a topic, and if it were, it wouldn't be a binary choice.
> I may have misunderstood,
You thoroughly misunderstood, no ambiguity. And no offense meant.
> ... but your thesis seems to be that markets are efficient, and no strategy can consistently beat the market over time.
No to the first (no one knows) but absolutely yes to the second -- no strategy can consistently beat the market over time. Isn't that obvious? Any strategy that consistently beat the market, and that could be expressed as a deterministic algorithm, and that was something other than a transient effect of no real value, could be used to drain the market of its capital in a matter of months -- and therefore it would be. The market would collapse and businesses would refuse to trust equities.
My point? If there really was such a strategy, it would demolish the market, meaning there would be no remaining market to beat consistently. The golden goose would lie dead.
Just think a bit more deeply.
> a random sequence of 39 heads is less probable than three sequences of 13 heads, each separated by some other unidentified, random sequences
Investors are not going for streaks. They are going for overall return. A better way to put it: what are the chances that over a 48-year timeframe, a random strategy will return ~20% annualized return compared to the market's ~10%? Ignore an arbitrary year limitation (why not measure monthly or daily return?). What is the chance a random strategy would outperform so long?
Or, a better question: what is a chance that a random strategy will be the best-performing investment over a 30-year period (1926-2011), and the subject of academic studies:
http://www.econ.yale.edu/~af227/pdf/Buffett's%20Alpha%20-%20...
Buffett is an existence proof that investment is skill, not chance. He created the 9th most valuable company in the world. You can believe that was entirely due to chance if you wish.
(On beating the market consistently: you don't have to crush the market to extract value. Certain strategies only work for certain amounts of capital. You can be an Apex predator and not drive the entire ecosystem into extinction.)
Easily answered. Let's say that a 10% return is the mean return, and one standard deviation is 5% -- just an example, and these numbers aren't real (although they could be established by asking everyone what their returns are).
So a return of 20% or better represents two standard deviations above the norm, or 2.2% of the investing population (this is a one-tailed distribution). How many investors will achieve that result in a large population? 2.2%. In a pool of a million investors, that's 22,000 people.
> Buffett is an existence proof that investment is skill, not chance.
With all respect, it's more accurate to say that your view of probability is an existence proof that many people don't understand statistics. Let me ask you -- do you understand how science works? Scientists don't say what you just said, ever. They say that the probability that this outcome resulted from chance is p ("p-value"), referring to the probability that the result arose because of chance. (My 2.2% probability above is a p-value.)
When the LHC scientists announced that they believed they might have detected the Higgs boson, did they say that their measurement constituted an "existence proof" that the Higgs was real? No, because they were scientists addressing educated people, they expressed their result in terms of a p-value -- p was the probability that their result came about because of chance, not the hypothesized particle.
A chance result isn't the last possibility that a scientist considers, it's the first. And no one who has been educated claims that a result that might have arisen by chance constitutes an "existence proof".
Quite correct, in fact, in an equities trading community the size of the present one, successes like Buffet's can and do result from chance. This is not to claim that any particular one does result from chance, only that this possibility is the first one that a scientist or statistician would need to consider.
That wasn't Buffets point; his point is that those who followed certain strategies of value investing consistently win and he gave examples. That's more than a random outcome.
"Consistently win" is obviously false. If someone had an actual method (not a random unexplained event) with a description, that could be tested and that could consistently beat market averages, it would surely be applied and the market would collapse. The market didn't collapse, so there is no such method.
How is that so hard to figure out? There is no winning strategy, no secrets of the winners. There are people who make more than others in equities, but not because of a describable, scientifically testable system.
When a scientist encounters a description like this, he always assumes a priori that it's chance. That agrees with the null hypothesis and Occam's razor. It also keeps people from selling him worthless investment books.
An analogy: humans have beaten computers at Go for decades. Is the human strategy random chance? It must be, since they cannot write down their algorithm so well that anyone else can be a Go master!
Go strategy (similar to investment strategy) cannot be perfectly laid down (though the general principles can), and yet the results are clear. Would you argue there is nobody inherently better at Go than anyone else? Even if they are the best performing Go-perfomer ever, who has beaten every computer system for decades and decades?
I'm not sure how much of a butt-kicking Buffett needs to perform against the market for you to believe there may be an element of skill at play.
You keep saying that, but it's full of assumptions that simply aren't true. Just because a successful method exists does not mean everyone can or will apply it nor does it mean the market will collapse.
> How is that so hard to figure out?
How it it so hard for you to see the absurdity of what you're saying?
> When a scientist encounters a description like this, he always assumes a priori that it's chance. That agrees with the null hypothesis and Occam's razor.
There's a reason scientists are in general not rich people. And I have much respect for scientists, but you're just being absurd.
Investigate a bit before just claiming it's nonsense because you're not giving it the thought you should, you're just presuming you're right and making absurd claims based on absurd assumptions that don't hold water.
http://www.businessinsider.com/warren-buffett-on-efficient-m...
So cash in. Since you think the EMH is "obviously untrue", you can drain the market of its capital based on your proven theory and your inside track on the truth.
In fact, plenty of evidence suggests that the EMH is true, but that the present equities market can't reliably demonstrate this fact because of widespread cheating.
I should add that, because of the nature of equities trading and markets in general, it's very doubtful that anyone will ever prove this issue one way or another in a scientific sense.
irrational != predictable ∴ markets cannot be accurately modelled
the EMH is true, but that the present equities market can't reliably demonstrate this fact because of widespread cheating
real markets != efficient and real markets != fair ∴ EMH is false for real markets
don't think so.
At what point do you say "The hypothesis that a professional investor with a published strategy who returns the best-performing fund in history appears to not be based on chance?".
Do you really think 2% of investors (1 in 50!) achieve 20% compound growth over 48 years? Do you know how many billions that is? (Buffet started with $100k and grew it to the 9th biggest company in the world. Where are the thousands of other investing billionaires?)
Do you really think beating the market long-term is a simple 1-time standard deviation computation? (I thought it was impossible, now 50% of people will consistently beat the market long term by any margin?)
Per the cited article, from trained economists:
"Buffett’s success has become the focal point of the debate on market efficiency that continues to be at the heart of financial economics. Efficient market academics suggest that his success may simply be luck, the happy winner of a coin-flipping contest as articulated by Michael Jensen at a famous 1984 conference at Columbia Business School celebrating the 50th anniversary of the book by Graham and Dodd (1934). Tests of this argument via a statistical analysis of the extremity of Buffett’s performance cannot fully resolve the issue."
This is false, and the fact that it's false has been proven and accepted for years.
http://www.nature.com/news/scientific-method-statistical-err...
Quote: "P values, the 'gold standard' of statistical validity, are not as reliable as many scientists assume."
http://bigthink.com/neurobonkers/the-statistical-significanc...
Quote: "P<0.05 is the figure you will often find printed on an academic paper, that is commonly (mis)understood as indicating that the findings have a one in twenty chance of being incorrect."
Also, your effort to estimate the probability of 39/47 is seriously flawed -- it depends on the distribution of successes and failures within the list.
> At what point do you say "The hypothesis that a professional investor with a published strategy who returns the best-performing fund in history appears to not be based on chance?".
Have you been reading my replies? The answer is never. The scientists at the LHC have acquired a p-value of 5-sigma for their observation of the Higgs, meaning the probability that their apparent detection of the Higgs arose from chance is 2.8 * 10^-7 (the "p-value"), but guess what? They are never going to say, "We detected the Higgs." Only ignorant science journalists say things like that, in the same way that ignorant stock market journalists say, "So-and-so has a reliable method for beating the market."
> Do you really think beating the market long-term is a simple 1-time standard deviation computation?
Yes, that's exactly what it is, given adequate data. And if that cannot be done, then economics has earned its reputation as a non-science.
> Do you really think 2% of investors (1 in 50!) achieve 20% compound growth over 48 years?
That was a hypothetical example, don't try to use it as an argument.
> I thought it was impossible, now 50% of people will consistently beat the market long term by any margin?
You will need to try to edit your posts to make them comprehensible. If you're asserting this viewpoint, I can prove that, for a random market with random buys and sells, with millions of investors, (a) 50% will do better than the average (uncontroversial), and (b) many of them will become rich by chance alone -- not just do better than the average, but become very rich with a small initial stake. Details here:
http://arachnoid.com/equities_myths/index.html#Market_Model
> Per the cited article, from trained economists ...
What? You're invoking the authority of "trained economists" to bolster your viewpoint? First, economists aren't scientists, second, an appeal to authority is a logical error, and third, your choice of quotation supports my view, not yours.
But I ask you to examine your position, and I shall help you do this by way of a reductio ad absurdum. Let's say there's a system for improving on average market returns, that it's something other than a chance outcome, and that it can be defined, tested, and thereby proven to exist in a scientific sense -- that it can be raised above the level of cocktail chatter.
1: Thesis: There is a way to trade securities that can be assured to reliably improve on average market returns, by means other than chance. The method can be written down, and therefore it can be tested objectively.
2: If (1) is correct, then anyone could take the proven, defined method, apply it to the market on a large scale, and drain it of all its capital in a short time.
3: But, notwithstanding the certainty of human greed and simple curiosity, (2) has not happened, anywhere, ever.
4: Therefore (1) is false.
Why is that definitive? Because science cannot be just descriptions ("Warren Buffett has an equities track record"), it requires explanations ("This is why Warren Buffett has an equities track record"). If there is no explanation, there's no science, and there's no point in conversations like this one.
Anyone can describe something in the environment, but the only interesting ideas are those accompanies by a testable explanation, a theory that purports to explain some aspect of nature, a theory subject to empirical test. And the outcome is interesting whether the theory turns out to be true or false -- we learn just as much from false scientific theories as true scientific theories.
And this is a classic debate between someone who understands science, and someone who doesn't.
The argument you need to address is whether it's possible for a person to consistently find new opportunities to arbitrage some inefficiency out of the market without relying on arguments that assume there's only one method and that it always works.
There's many things to say, but just consider that you believe beating the market over a 47-year time period comes down to a standard-deviation calculation where half will, half won't. Just ruminate on that. If you can't see the absurdity of it, I'm not sure what to say.