First of all, it’s important to remember that the S&P 500 is just a bundle of a bunch of companies, so when it’s up that means that the constituent companies are growing in value. So the question can be restated as, “why, in the aggregate, do large cap companies, like the ones found in the S&P 500, tend to grow in value in the long term?” Here are a few answers:
This seems like a pretty solid candidate for answering our question. Technological progress has continued unabated for the lifetime of the S&P 500 (and for the almost entirely of human history before that), resulting in ever increasing quality of life, continually creating value for the world.
Patents
For evidence of this theory, we can look at the number of patents issued per year to try to represent technological progress. That graph looks like this:
Let’s compare the percent change of the number of utility patents issued and compare it to the percent change of the S&P 500 for the last 65 years:
Lets smooth that out by graphing the 6 year moving average for the percent change in the S&P 500 and number of utility patents issued:
From this graph, we can see that changes in the S&P 500 and the number of patents issued seem to be somewhat related. It’s important to remember that neither metric is a completely independent variable. It’s logical to assume, and this graph seems to indicate, that slowing technological progress would hurt the economy just as a weak economy would lead fewer people to apply for patents. The question is to what degree each metric depends on the other. That’s a difficult question to answer because the forces affecting that graph are numerous and complex, so it’s unclear whether a change in the S&P 500 on the graph is due to a change in the number of patents or some other change in the world that is affecting one or both of them. However, what is clear is that there is a strong relationship between the number of patents filed and the performance of the S&P 500. To emphasize their relationship, I graphed the price of the S&P 500 and the number of patents issued from 1950 to 2015. Here’s what that looked like:
Also, the Pearson product-moment correlation coefficient of the patents issued and the S&P 500 is 0.956 — that’s pretty meaningful.
Another curiosity worth mentioning is how there are so many points clumped together in the bottom left corner of the graph. This is because, for the majority of the years between 1950 and 2015, the S&P 500 was below 600 and the number of patents being issued was barely above 100,000. The patents and the S&P 500 have grown exponentially since then. This brings us to the next part about technological progress.
Exponential Growth
Those interested in future studies will likely have read Ray Kurzweil’s 2001 essay, The Law of Accelerating Returns in which he argues that technological change is exponential. If technology progresses exponentially and our economy is based on technological progress, we should expect our economy to similarly grow exponentially.
In his book on artificial intelligence, Superintelligence, Nick Bostrom had this to say of exponential economic growth:
The economist Robin Hanson estimates, based on historical economic and population data, a characteristic world economy doubling time for Pleistocene hunter-gatherer society of 224,000 years; for farming society, 909 years; and for industrial society, 6.3 years.
If technology is the driving force behind economic growth, and technology improves exponentially, we would expect the economy to grow exponentially as well. Throughout human history, this seems to have been the case. To see if this holds true in modern times, lets graph US GDP per capita for 1871 to 2009 and put an exponential trend line over it:
As we would expect, the US economy grows exponentially. This is strong evidence to support the idea that technological progress is the root of economic growth
As technology continually improves, so does productivity. These two are related concepts because it is technology that improves productivity — a worker with Excel and an internet connection can perform far more work in an hour than someone without these tools.
Nick Bostrom also addresses exponential productivity improvement in Superintelligence:
A few hundred years ago, in early human (or hominid) prehistory, growth was so slow that it took on the order of one million years for human productive capacity to increase sufficiently to sustain an additional one million individuals living at subsistence level. By 5000 BC, following the Agricultural Revolution, the rate of growth had increased to the point where the same amount of growth took just two centuries. Today, following the Industrial Revolution, the world economy grows on average by that amount every ninety minutes.
Shawn Sprague discusses the relationship between labor productivity and economic growth in his essay in Beyond the Numbers. He points out that the total number of labor hours worked across the economy hasn’t changed in recent years despite a large increase in the US population, in businesses operating in the US, and in total output. In fact, Sprague cites that from 1998 to 2013, there was no growth in the number of labor hours worked, but a 42% growth, $3.5 trillion, in output. This would seem impossible — in order to produce more, one must work more — except for the fact that productivity also grew over that same period. We are able to work less and produce more, creating more value for businesses and improving our own lives, thanks to our growing productive capacity.
The consistent growth we have experienced in productivity is a strong driving force behind our economic growth and that of the S&P 500.
A company that can sell its products to one billion people is more valuable than a company that can sell its products to only one million people. In this way, increasing populations create value. So population growth can spur economic growth and growth of the S&P 500.
Here’s a graph of the total US population from 1790 to 2000. It shows the ceaseless historical exponential growth of the population of the United States.
And remember that Nick Bostrom quote about how improvements in productivity allowed for the the economy to grow to support a greater population living at subsistence level? This is all relating back to technology.
Population growth may also be related to improvements in technology and productivity in another way. This is because, if intelligence is on a bell curve — which it is — and you have a small population, you won’t have many people who are incredibly intelligent, because such a high IQ would be very rare and unlikely to be found in a small population. As populations increase and the “genetic dice” are rolled more frequently, there will be more incredibly intelligent people born, leading to more innovation increased prosperity for the entire species. I don’t have a graph for this, so take it with a grain of salt, but I think that the idea has merit.
In my mind, this is an incredibly overlooked feature of the S&P 500:
The S&P 500 does not exist in nature. Its constituent companies have been selected subjectively based on the expectation that those companies will grow in value.
In other words, many people seem to think of the index as immortal and unchanging, but the companies that make up the S&P 500 are hand picked based on subjective criteria, including “financial viability.” When a company is no longer thought of as “financially viable” in the eyes of those who manage the index, it is replaced by another company. This has happened 1,186 times between 1963 and 2014. So the S&P 500 continues to deliver returns because under performing companies are cut. This is perhaps a far less important factor, but is worth mentioning because of how underappreciated this fact is.
So if we’ve established that the S&P 500 and US economic growth is largely dependent on technology, there is one very important resultant implication that should be addressed. This is that, should our species or our country fail for any sufficient length of time to continue to innovate, there will be disastrous consequences. The principle that retirement plans, pension funds, and more rely on, a generally accepted rule that has always been true — that the S&P 500 and large cap stocks in the aggregate tend to grow in value over the long term — will cease to hold true. In my view, this would certainly precipitate a severe economic depression.
But this perilous arrangement is also at the heart of our potential. Our world and our economy is geared towards growth. We understand and expect that the world we will experience in a decade will be far better than that of today, and we know that the world we leave to those who come after us will be profoundly different from our own. Their world will be safer, more connected, and more unified; they will spend more of their time in leisure, they will be healthier, and the greatest problems of our time will be solved — that solution being taken for granted as a fact of life in their world. It was not only curiosity, but an unselfish love for those who would follow them, that compelled our grandparents’ generation to develop penicillin, the microwave, the bikini, and duct tape. Each generation’s iterative improvements to our species, starting with our hunter-gatherer ancestors developing language and control of fire, brought us here.
“If I have seen further, it is by standing on the shoulders of giants.” -Isaac Newton
With continued, bold investments in science and technology, we can carry this torch further, improving the condition of our species, bringing ourselves greater prosperity, and making good on the covenant we have with our descendants.
