Investor sentiment and the near-term stock market

https://doi.org/10.1016/j.jempfin.2002.12.001Get rights and content

Abstract

We investigate investor sentiment and its relation to near-term stock market returns. We find that many commonly cited indirect measures of sentiment are related to direct measures (surveys) of investor sentiment. However, past market returns are also an important determinant of sentiment. Although sentiment levels and changes are strongly correlated with contemporaneous market returns, our tests show that sentiment has little predictive power for near-term future stock returns. Finally, our evidence does not support the conventional wisdom that sentiment primarily affects individual investors and small stocks.

Introduction

For decades, an obvious schism has divided the field of finance: The success of the efficient market hypothesis (EMH) in explaining the lack of predictability in liquid asset returns has clashed with the traditional “search for value” undertaken by many finance practitioners. Only since the mid-1980s has there been a serious attempt to explore the possibility that liquid financial markets are not always as orderly as might be suggested by the efficient market advocates. For example, as the “noise trader” theories of Black (1986) and DeLong et al. (1990a) suggest, if some investors trade on a “noisy” signal that is unrelated to fundamentals, then asset prices will deviate from their intrinsic value. Unfortunately, most evidence supporting the noise trader theory is, at best, controversial.1 Other recent empirical work also documents stock market anomalies such as market under- and over-reaction.2

More recently, attempts have been made to measure to what extent historical stock prices have reflected underlying fundamentals. Lee et al. (1999) calculate the intrinsic value for the Dow Jones Industrial Average (DJIA) from 1963 to 1996 and are able to predict subsequent market returns. Bakshi and Chen (2001) propose a model for the price of stocks (and stock indices) and present estimates showing persistent deviations from fair value for Dow 30 stocks, several technology stocks, and the Standard and Poor's 500 Index.

It is well known that market imperfections lead to observed prices deviating from underlying value (see Grossman and Stiglitz, 1980). Perhaps, the most important of these frictions for large US stocks is the costly nature of value-relevant information. Because so much of the information regarding the distribution of future stock returns is costly (or even impossible) to obtain and even then subject to interpretation, prices almost certainly trade inside a band around their value given perfect information. This judgment is supported by many empirical findings. For example, Seyhun (1986) shows that corporate insiders trading in their own stock earn superior returns. Donaldson and Kim (1993) find the DJIA is subject to “support” and “resistance” levels at multiples of 100. Siegel (1992) shows that neither changes in interest rates nor changes in future earnings account for the dramatic valuation changes around the October 1987 crash. He concludes that shifts in investor sentiment are correlated with market returns around the crash.

None of this is news to finance practitioners. It is hard to open a financial paper or turn on a market news program without hearing of an analyst having turned bullish or bearish on the market. In fact, for as long as markets have existed, pundits have had opinions concerning their relative value. Included in this group are esteemed economists and respected government officials. For example, Adam Smith (1776, p. 24) noted that, “The actual price at which any commodity is sold is called its market price. It may be either above, or below, or exactly the same as its natural price.” More recently, Alan Greenspan (who spent 30 years on Wall Street) made news with his “irrational exuberance” speech, voicing his opinion that stock prices were too high.

In short, market watchers and participants seem to believe in “sentiment.” But, what exactly is sentiment? Intuitively, sentiment represents the expectations of market participants relative to a norm: a bullish (bearish) investor expects returns to be above (below) average, whatever “average” may be. While there is no doubt such expectations exist,3 the primary goal of this research is to investigate the role (if any) of sentiment in the price formation process for stock portfolios.4 We conduct our analysis in two main steps.

First, we determine the relations between survey data for investor sentiment and other commonly cited “sentiment measures” such as the advance-decline ratio, short interest, and closed-end fund discounts. Many of these measures contain related information. Consequently, we employ the Kalman filter and principal component analysis as means of extracting composite unobserved sentiment measures. We also separate potential sentiment measures into two groups, institutional and individual, to see if there are different classes of investor sentiment.

Next, we examine the ability of these sentiment measures to predict returns. Specifically, we explore the bi-directional relation between investor sentiment and near-term stock returns in a vector autoregression (VAR) framework using the composite sentiment measures discussed above.

Overall, our findings are consistent with many of the anecdotes regarding investor sentiment. Specifically, we document strong relations between many disparate measures of investor sentiment. This probably explains why certain indicators have developed a reputation as sentiment measures. Furthermore, we document that changes in the survey and our composite measures of investor sentiment are highly correlated with contemporaneous market returns. The empirical sticking point is that this correlation does not directly reveal the causal relation between sentiment and the market.5 The more sophisticated VAR analysis reveals that market returns clearly cause future changes in sentiment. However, very little evidence suggests sentiment causes subsequent market returns. This is bad news for investors trying to use sentiment measures for short-term market timing. It appears that such strategies are not profitable during our reasonably long sample period.

Our paper is not the first to explore the role of investor sentiment in the stock market, but it is the most comprehensive study to date. A series of papers has debated about closed-end fund discounts as a measure of sentiment. Lee et al. (1991), Swaminathan (1996), and Neal and Wheatley (1998) claim closed-end fund discounts measure investor sentiment, while Chen et al. (1993) and Elton et al. (1998) provide evidence to the contrary. Neal and Wheatley (1998) also find that net mutual fund redemptions are useful in predicting the size premium. Brown (1999) shows that sentiment is closely related to closed-end fund price volatility. Barber (1994) shows that the behavior of Prime and Score premiums is consistent with a noise trader model. Finally, Clarke and Statman (1998), Otoo (1999), Simon and Wiggins (1999), and Fisher and Statman (2000) examine the usefulness of a variety of technical variables in predicting short-horizon market returns. Our analysis expands upon these and other papers by considering a more comprehensive set of sentiment proxies, using direct (survey) data on sentiment, examining the (statistical) causal relation among the variables, and finally by investigating the relation between sentiment and subsequent market returns. In a companion paper, Brown and Cliff (2004) explore long-run effects and find that optimism is associated with overvaluation and low subsequent returns as the valuation level returns to its intrinsic value.

Section snippets

Motivation

We start our analysis by providing an informal discussion of the ways in which we expect sentiment to affect market valuations and returns. These predictions provide a framework for the empirical analysis and interpretation of the results. A number of researchers, such as Grossman and Stiglitz (1980), Black (1986), DeLong et al. (1990a), Campbell and Kyle (1993), Barberis et al. (1998), Daniel et al. (1998), and Hong and Stein (1999) have more formally modeled the role of sentiment or investor

Data

In implementing our analysis, we have chosen to concentrate on market aggregates instead of individual stocks. While the effects of sentiment will aggregate across stocks to the market level, we run the risk that roughly as many stocks are affected by bullish sentiment as bearish sentiment and thus the aggregate affect is negligible. Our choice of market aggregates is primarily driven by data limitations. Many of the measures we examine are available only for the market as a whole (i.e., the

Methodology and results

The contemporaneous relations between changes in many measures of investor sentiment and market returns are strong. For example, correlations between survey measures and returns reported in Table 2 are consistently positive and significantly different from zero especially at the monthly frequency. Univariate regressions (not tabulated here) with returns as the dependent variable consistently reveal statistically and economically significant coefficients on changes in sentiment measures.18

Conclusions

In summary, we have demonstrated that surveys measuring investor sentiment are related to other popular measures of investor sentiment and recent stock market returns. By using signal extraction techniques, we have been able to isolate common features of these indicators for a long monthly time period (34 years) and a shorter weekly time period (11 years). In the weekly sample, we have generated two separate measures that we believe represent institutional and individual investor sentiment.

For

Acknowledgements

Funding from the Cato Center for Applied Business Research is gratefully acknowledged. We thank Kent Daniel, Greg Kadlec, Steve Slezak, Kent Womack, seminar participants at the University of North Carolina, Virginia Tech, the 1999 Western Finance Association meeting (especially the discussant, Bhaskaran Swaminathan), the 1999 Financial Management Association meeting, and the 2000 Batten Young Scholars Conference at William and Mary for their comments and suggestions. We also thank Greg Kadlec

References (43)

  • Z. Chen et al.

    Portfolio performance measurement: theory and applications

    Review of Financial Studies

    (1996)
  • N. Chen et al.

    Are the discounts on closed-end funds a sentiment index?

    Journal of Finance

    (1993)
  • R.G. Clarke et al.

    Bullish or bearish?

    Financial Analysts Journal

    (1998)
  • K. Daniel et al.

    Investor psychology and security market under- and overreactions

    Journal of Finance

    (1998)
  • J.B. DeLong et al.

    The size and incidence of the losses from noise trading

    Journal of Finance

    (1989)
  • J.B. DeLong et al.

    Noise trader risk in financial markets

    Journal of Political Economy

    (1990)
  • DeLong, J.B., Shleifer, A., Summers, L.H., Waldmann, R.J., 1990b. The survival of noise traders in financial markets....
  • R.G. Donaldson et al.

    Price barriers in the Dow Jones industrial average

    Journal of Financial and Quantitative Analysis

    (1993)
  • E.J. Elton et al.

    Do investors care about sentiment?

    Journal of Business

    (1998)
  • K.L. Fisher et al.

    Investor sentiment and stock returns

    Financial Analysts

    (2000)
  • N.G. Fosback

    Stock Market Logic: A Sophisticated Approach to Profits on Wall Street

    (1993)
  • Cited by (0)

    View full text