Historical index constituents and component changes

The problem this dataset exists to solve

A backtest built on today's index membership is testing a strategy that had access to information nobody had at the time. This is survivorship bias, and on index data it is unusually severe, because index membership is not random. Companies leave an index for reasons: they shrink, they are acquired, they are delisted, they fail. Companies join for reasons too: they have grown, usually after a long run of good performance.

Take today's constituent list and run it backwards and both effects work in your favour. The companies that collapsed out of the index are simply absent from the history. The companies that joined after a strong run bring that run with them into the sample. The resulting backtest does not measure a strategy; it measures a selection rule that could not have been applied.

The distortion is largest exactly where it matters most — in downturns, when removals cluster. A test of how a strategy behaved in 2001 or 2008 using the current membership list is a test on the survivors of those years, which is close to the opposite of what the test is for.

Two reasons point-in-time data is harder than it looks

Reconstruction is not the same as recording. Membership can sometimes be inferred backwards from surviving records, but inference makes choices, and those choices are invisible to whoever uses the file afterwards. A dataset compiled from contemporaneous records carries the information as it was, including the parts that later turned out to be wrong.

Identifiers move. Companies change name, change ticker, merge, spin off and re-list. A membership history that tracks only the ticker will quietly attach one company's history to another's. The joins are where most of the work in this kind of dataset actually sits.

The index effect: the most-studied question you cannot test without this data

The best-known use of constituent change data is the index effect — the abnormal return a stock earns between the announcement that it will join a major index and the day the change takes effect, as index funds are obliged to buy it.

It is also the clearest illustration of why the history matters rather than a recent sample. In The Disappearing Index Effect, a study built on this data, Greenwood and Sammon put the average abnormal announcement return for U.S. additions at around 3.4% in the 1980s, rising to roughly 7.4% in the 1990s, and falling to about 1.0% and statistically insignificant by the 2010s — a decline they attribute mainly to a collapse in the price impact per unit of index-driven demand, with index migrations and the increasing predictability of changes contributing as well.

Three things follow, and all three need the announcement date rather than the effective date — which this database records for the most widely tracked indices:

• A result measured on one decade does not transfer to another. The effect was largest in the decade when passive ownership was growing fastest, not the decade when it was largest in absolute terms.
• The gap between announcement and effect is where the return sits. A dataset carrying only effective dates cannot see it at all.
• Anomalies decay once they are known. The index effect is now a fairly well-behaved example of that, which is itself a finding — and one nobody could have established without four decades of membership changes recorded as they happened.

How index membership is actually decided

The distinction matters more than it sounds. For a rule-based index, a researcher can often reconstruct what the membership should have been. For a committee-driven one, there is no rule to apply — only the record of what the committee actually did.

Indices change more often, and less tidily, than people expect

Scheduled reviews are typically quarterly or annual. Unscheduled changes happen whenever a constituent is acquired, merges, is delisted or fails, and those arrive without regard to the calendar.

One consequence worth stating plainly, because it surprises people: an index named for a number of companies rarely contains exactly that number. The Russell 3000 usually holds fewer than 3,000 names. The count drifts between reconstitutions as companies disappear and are not immediately replaced.

Weighting is a third thing the membership list does not tell you

Knowing who was in the index is not the same as knowing what the index was. Three schemes dominate:

• Market capitalisation weighting — the most common. Position size follows company size, so the largest constituents dominate index behaviour.
• Free-float market capitalisation weighting — the same, with restricted, closely held and government-held shares excluded, so the weight reflects what is actually available to buy.
• Equal weighting — every constituent carries the same weight, which requires frequent rebalancing and gives small constituents influence far beyond their size.

Under capitalisation weighting a handful of very large companies can account for a large share of a broad index, and the index's behaviour becomes substantially a statement about them. Reproducing an index's historical return therefore needs the weights, not only the names.

Who uses this

Quantitative researchers and factor investors need point-in-time membership because everything else they do rests on it. A factor portfolio constructed from a contaminated universe produces a clean-looking result that cannot be traded.

Academic researchers study index additions and deletions as a natural experiment: a demand shock with no accompanying news about the company's fundamentals. That design only works with announcement dates and effective dates held separately.

Asset managers and ETF analysts need the change record for attribution, for tracking-error analysis and for compliance — reconstructing what a mandate held on a date, and why.