The theoretical U.S. “market” that is so often spoken of by stock investors is really just a convenient fiction. What does the term really mean? The difficulty with defining it can be seen as soon as we ask ourselves which stocks the market should include and which it should exclude. Should it consist of only U.S. stocks or are there reasons to include foreign companies that do a lot of business in the U.S.? If we decide to only include those from the United States, then are stocks to be counted only if they are headquartered in the U.S. or is trading on a U.S. exchange good enough? Should the market include only larger companies that we are probably more likely to buy, or should it include companies of varying sizes including penny stocks? Should stocks be limited to those that trade on major exchanges (like the NYSE and NASDAQ) or should it also include stocks from lesser exchanges (like the American Exchange) or even pink sheet and over the counter stocks?
Suppose, for the sake of argument, that we somehow decide which stocks to include and which to exclude in our definition of the market (we might call this our “universe” of stocks). Matters get further complicated when we try to figure out the performance of our market over a given time period (for example, if we want to answer the question of whether the market was “up” or “down” in the past week). Should the market’s percentage return be defined as the average percentage return of all of our chosen chosen stocks? Or could we use the median return of these stocks instead? Or perhaps we should take a weighted average of each of the stock returns, where the amount that each of these returns counts is proportional to the market capitalization (a.k.a. market cap, which is the number of shares outstanding times the share price) of each stock (as in the Russell 1000 index). On the other hand, rather than weighting by market cap, maybe we should weight each stocks return by its float, which adjusts the market cap to include only those shares that are publicly available for trading (as in the S&P 500 index). One might even suggest that we weight the returns of each stock by the stock’s price (as in the Dow Jones Industrial Average).
How is one to decide between all of these different selection and weighting alternatives? Some popular choices to use as our definition for the U.S. market are:
1. The Dow Jones Industrial Average (a.k.a. The Dow) : The Dow is an infrequently updated group of thirty stocks from U.S. exchanges (but never utility or transportation companies) that are selected by the editors of the Wall Street Journal. The returns of these stocks are weighted proportionally to each stock’s share price. The Dow is probably the most frequently quoted stock market index in the world today, as well as the oldest market indicator (started in 1896). Unfortunately, it is also one of the worst market indexes in existence, in part due to the technological limitations at the time of its creation. Although highly correlated with other indices like the S&P 500, the fact that the Dow consists of only 30 stocks means that at times it can deviate substantially from broader market indices due to the volatility of individual stocks. What’s more, the constituents of the Dow are not generally representative of the industry breakdown of the stock market as a whole or of the United States economy. Finally, its method of weighting stock returns by share price is nonsensical: the share price is essentially meaningless, heavily influenced as it is by irrelevant factors such as the number of stock splits a company has had and the price at which shares were initially offered. It is time for America to abandon this subpar market index. The only really legitimate use that the Dow has is for looking at stock market characteristics over very long time periods (owning to its very early creation date). Unfortunately, as things stand today, when U.S. investors say “the market” they are typically referring either to the Dow or the S&P 500.
2. The S&P 500 : A float (formerly market cap) weighted index of five hundred of the largest market cap stocks that trade on the NYSE and NASDAQ exchanges. These stocks are chosen by a committee at Standard and Poor’s with the goal that they are representative of the industry breakdown of the broader stock market, and also that each of the stocks has sufficient liquidity, “financial viability”, and other characteristics. This index is probably the second most often quoted after the Dow Jones, and dates back to 1957. The S&P makes a lot more sense than the Dow, given its float weighting (which, roughly speaking, counts the returns of each stock based on how much the stock’s tradable shares are worth as a fraction of the value of all tradable shares of stocks in the index). This makes much more sense than the Dow’s price weighting, and can be interpreted as trying to approximate the average performance of a randomly placed dollar placed in the market. What’s more, the S&P 500 consists of five hundred rather than thirty stocks, which makes it better able to capture broad market changes and less susceptible to individual stock volatility. Unfortunately, its choice of which five hundred stocks to include is somewhat arbitrary from the point of view of most investors. In terms of total market capitalization, the S&P 500 represents “approximately 75% of the U.S. equities market.”
3. The Wilshire 5000 : A market cap (or in one version, float) weighted index of all companies with U.S. based headquarters that trade actively on a U.S. exchange and have “readily available price data”. This index has some advantages over the S&P 500, in that it is much broader (containing well over five thousand stocks), including a great number of smaller companies that the S&P ignores, and therefore may work better as an indicator of the entire market. It should be noted, however, that since the Wilshire 5000 is market cap weighted, the largest stocks count far more than the smallest ones, and so as it turns out the movement of the S&P 500 and Wilshire 5000 are pretty similar in practice, both dominated by their largest constituents. That being said, sometimes the performance of small market cap companies can diverge substantially from large cap companies, which alone can lead to performance differences between the two indices. One draw back of the Wilshire 5000 is that highly illiquid stocks that have very large erratic (and sometimes practically meaningless) movements in their stock prices can add volatility to the index. Another problem that the Wilshire 5000 suffers is that it contains so many different stocks that it can be difficult to replicate its actually return. Buying the appropriate number of shares of each of its constituents is difficult, both because of the large number of stocks involved, and because of the potentially low trading volumes that these stocks may have. This can be be a substantial draw back, as there are many investors who are interested in being able to achieve the “market” return (or, at least, a very close approximation thereof).
4. The Russell 3000 and Russell 1000 : These market cap weighted market indices contain the three-thousand or one-thousand stocks (respectively) with largest market caps that have headquarters in the U.S.
5. The Russell 2000 and S&P 600: These market cap weighted indices are designed to track the results of the smaller companies in the market (and are generally known as small cap indices). The Russell 2000 consists of the two-thousand smallest stocks from the Russell 3000, and represents approximately 8% of the Russell 3000′s total market capitalization. The S&P 600 consists of six hundred stocks that are selected in a manner similar to that of the S&P 500, but which are limited to have market caps between $200 million and $1 billion, covering approximately 3% of the total market cap of U.S. equities markets. For some investors, the Russell 2000 or S&P 600 may be more reasonable benchmarks to compare their performance against than large cap indices like the S&P 500. Since the S&P 500 is market cap weighted, and the largest market cap stocks are so very much larger than the average stock, the S&P 500 has a tendency to have its movements be dominated by a small number of extremely large companies. For investors that tend to buy smaller companies, the performance of the S&P 500 simply may not be that relevant to them. In practice, small cap and large cap indices can diverge quite substantially in their performance, especially over periods on the order of a few months. It is worth noting that since most stocks in the market are not as large as those in the S&P 500, the small cap indices may do a better job of capturing the behavior of more “average” stocks. On the other hand though, since people tend to think of the large cap indices like the S&P 500 as being “the market”, the large cap indices may do a better job of capturing market sentiment than small cap indices.
6. S&P Equal Weight Index : This index is constructed just like the S&P 500, but instead of weighting stock returns by market cap, the returns of all the constituent stocks are simply averaged together. This leads to a different sector allocation than the S&P 500, and performance that can differ fairly substantially. This benchmark may be appropriate for, say, investors who are limited to invest in only stocks with a large amount of liquidity, but who (once this constraint is satisfied) are not more likely to prefer larger stocks over smaller ones. Equal weight indices effectively capture the average performance that would be achieved if stocks were picks at random from the chosen stock universe.
As you can see, there are a large number of different indices that are commonly used, each of which has at least some claim for being called “the market” (at least, from the point of different individual investors). Ultimately, the best choice of which index to use depends on one’s goals. In many cases market indices are used to benchmark an investor’s performance, essentially determining whether she has performed better than her competition or asset class. In that case, the choice of which stocks to include in your definition of the market should effectively represent the universe of assets from which the investor has decided (in advance) to select their investments. For example, if upon founding their business, an investor has determined to only buy large market cap companies from South East Asia, then the market for that investor should consist only of stocks that meet those specific criteria.
As far as how stock returns should be weighted when constructing an index for benchmarking an investor, the two most natural choices are market cap weighting and equal weighting. Market cap weighting produces an approximate measurement of the average performance of each dollar that is allocated to a universe of stocks, whereas equal weighting produces a measurement of the average performance of each of the stocks themselves. If we want to answer the question of whether an investor’s capital outperformed the average return of a random dollar allocated to her universe of stocks then market cap weighting is the natural choice, whereas if we want to see whether an investor’s performance beat how one would do if they selected stocks randomly then equal weighting makes sense. Unfortunately, since smaller market cap companies tend to have larger, more random and more erratic swings in price, equal weighting stocks (especially fairly illiquid ones) can lead to very volatile (and sometimes essentially meaningless) market indices. The S&P Equal Weight Index avoids these problems to a large degree by limiting its universe to only very large companies, which tend to be very liquid and hence less prone to large, sudden fluctuations in stock price.
It is worth noting that while market cap weighting does have a certain intuitive appeal, it does not, by any means, do a perfect job of measuring the average performance of each dollar in a market place. One major problem is that market cap is not a fully satisfying measure of how much money is currently invested in a given stock. The problem arises because the price that we can sell one share of a stock for today (which is the price used in the computation of market cap) is not necessarily the price per share at which we could sell as many shares as we own (in fact, if we own a large block of shares we will often have to move the price slightly below the current market price in order to get those shares sold). This being said, if there is a better way than market cap weighting to measure the average dollar performance in a universe of stocks, I am not familiar with it.
Beyond benchmarking purposes, people frequently use measures of market performance as economic indicators, essentially treating the stock market as a proxy for business as a whole, or as a measure of the average performance of a dollar invested in the stock market. In these cases, it is generally desirable to use indices like the Wilshire 5000 because it consists of a very large number of companies and is market cap weighted. Since smaller companies do account for a significant portion of both business activity and stock market investment in the U.S., narrower indices like the S&P 500 may not be adequate for these economic indicator purposes.
In conclusion, while there are many possible ways that we may define the market, some of which are quite similar to each other in construction or performance, we should attempt to find the best possible definition to suit the particular task at hand. Ultimately, with so many options available, there is no reason to settle for a market index that is only “good enough”, since better decisions can be made using an index that is “just right”.