A Quanty Way To Analyze Sentiment, Momentum and Breadth in Equities

Backtesting some indicators that reveal the underlying dynamics of an equity index (code and historical S&P 500 constituents data included).

Throughout the day, the financial media reports on price developments in the stock market, among others. But, they only focus on stock indices and any stocks that are part of the day’s news, such as those reporting earnings that day. It is like standing on the edge of a pond all day, and taking note of the changes on the surface of the water, and any creatures that pop out of the water; there’s a whole world of things beneath the surface that you are not privy to.

I am not critiquing the financial press; I’m just saying that if you want to know what’s really going on with stocks—like what the market breadth is like, what is driving the index returns, what is the sentiment and momentum like, and more—you will need to dig around some more for that information. The weekly market report published in this newsletter is one of the places you can look to for that kind of analysis.

One of the things I want to include in that report is a section about this kind of stuff. While trying to figure out how to go about it, I decided to do some backtests of the various indicators I will be using so that I know what to include and what not to, and to learn what to look out for when using these indicators.

There are many indicators used to analyze the underlying dynamics of an index. My plan for the weekly market report is to start small and then add or remove them depending on their proven usefulness. In this post, I focus on two main indicators of sentiment and momentum, and market breadth.

What I Did

One of the reasons you will not find much commentary or research on what index members are doing is that keeping track of index members isn’t easy. Some paid services offer this information, but subscription prices can be high. One way to find out is to do a bit of web scraping. In this case, the web scraping involved going to the S&P 500 Wikipedia page, getting the current members, getting all the added and removed stocks, and reconstructing the historical constituents. I used R for that.

Due to de-listings, mergers, and acquisitions, there were a lot of missing tickers when I went to download price data. In 1990 (how far back I went), there were 87 and in Jan 2021, there were 24. I decided that 24 was not that bad and that any more would affect the strength of the indicators, so I started the backtests from Jan 2021 to Oct 2024. Is it long enough for the results to be statistically meaningful? I don’t know, but I would rather have a short period when the indicator signals are reliable than a long one when they are not. Plus, the objective here is not to get hard and fast rules to apply to these indicators but to understand how they work. With the list of historical constituents and the corresponding historical prices, I calculated the percentage of constituents above their 50-DMA, 100-DMA, and 200-DMA, and the number of constituents outperforming the index.

What I like about the first indicator is that it does not rely on the actual returns of the index components. It tells you something about sentiment and momentum in stocks in a way that is comparable over arbitrary periods, and robust to outliers. It also says something about market breadth. The second one is good for market breadth but is influenced by outliers.

I’ve seen some analysts use indicators like average distance from a DMA (how much a stock needs to move to reach a daily moving average) and the average distance from the index (how much a stock needs to gain to catch up to the index’s returns). These indicators are sensitive to outliers, and averages murk the picture; we are searching for clarity. Two indicators that I think are also good and will focus on later on are the percentage of stocks in a bull/bear market (number of stocks in a new 52-week high/low) and advance/decline (percentage of stocks going up or down).

---

Editor’s note: I’ve been working on a comprehensive backtest of CFTC data. It got too big that it wasn’t feasible for me to write the results, so I decided to publish them on a web app. I can’t afford to hire help rn so I have to do it all myself and that’s why it’s taking forever. But it’s worth the wait :)

---

For the backtests, the goal was to see if the indicators gave a reliable signal about the index’s direction; not to outperform the benchmark (buy-and-hold). Therefore, there are backtesting best practices I didn’t follow including transaction fees and stop losses. I’ll also say that there is a small amount of survivorship bias here because the stocks that were used to calculate the indicators are the ones that are still around today, but from 2021, the number of missing stocks kept declining to about 7 in Q4 2022 which I think reduces this bias. These are the important things to remember when you look at the backtest results.

Overview of the Selected Indicators

As I said, the two indicators we focus on are market breadth, and sentiment and momentum in stocks. Market breadth is the number of index components participating in its advance or decline. A naive way to calculate this is by just tallying the stocks going up or down and checking if the index went up or down too, but indices are weighted averages so this might not be the best approach. A better way is to look at the number of stocks outperforming the index. These are the stocks that are in the driver’s seat. Breadth widens when more stocks outperform the index, and contracts when fewer stocks do. However, underperformers can significantly influence this indicator because if there are a few big losers, they will pull the index down and many other members will outperform it. That’s why you should look at several indicators together to get the full picture.

Sentiment and momentum don’t have to go together—they are different things! Sentiment is how investors feel about the market (bullish/bearish) and momentum is about the strength of a trend. For more assets, it is possible for investors to be bearish and stocks to have bullish momentum, and vice versa. But in this case, the percentage of stocks above a moving average gives some information about the overall sentiment and momentum because you are aggregating over all index members.

Percentage of Constituents Above a Moving Average

You would think this indicator is lagging because it involves moving averages, but it isn’t. It moves with the index. How much it moves gives you a signal about the sentiment and momentum in stocks. Here we have the S&P 500 this year, and the percentage of constituents above the 50, 100, and 200 DMAs.

The percentage of stocks above their 50-dma indicator is more volatile which is expected. The percentage of stocks above their 100-dma moves within a narrower band but it moves up and down. The percentage of stocks above their 200-dma has been quite high all year, spending most of the year above 70%.

Here is the same chart going all the way back to November 2021.

Most of the time when the percentage of stocks above their 50-DMA went above 80%, it would top out and the market would turn. However, in January 2024, this didn’t happen. The percentage of stocks above their 50-DMA topped out at 92.5% on January 2nd, and then dropped and have never been to those levels since, even though the market has seen going higher and higher. This was the bullish sentiment extreme of the year after stocks started rising in October 2023 in anticipation of rate cuts. All three indicators rose from their yearly lows from Oct ‘23 to Jan ‘24, however, the number of stocks above their 200-DMA remained high all year which shows that momentum was bullish.

So, it seems that they all capture slightly different things. The percentage of stocks above their 50-DMA is more of a sentiment indicator; when it reaches extreme levels, a turnaround is usually imminent, but not if there is momentum to the opposite side which is captured more by the other two—especially the percentage of stocks above their 200-DMA.

You can see this if you look at the correlation of the indicators and the prices. The percentage of stocks above their 50 and 100 DMAs have a correlation of 99%. They are the same but one moves more dramatically and both have a weak positive correlation to prices—both are more sentiment than momentum indicators. The percentage of stocks above 200-DMA has a higher correlation to the prices than the other two and a higher (and equal) correlation to the other two indicators than prices—it is more of a momentum than a sentiment indicator.

Percentage of Outperforming Constituents

This indicator is difficult to read because it varies wildly from one trading day to the next. Here is the chart along with S&P 500 prices.

If you zoom in on the Oct ‘23 to April ‘24 rally, here is how it looks.

Not readable. Compare that to the percentage of stocks above their DMA over the same period below.

Despite the percentage of outperforming constituents indicator not being too readable, backtest results show that it gives good signals at the extremes that are more timely than the later set of indicators (1-3 days before the market turn).

Backtests