Data Reference Overview¶

Coverage¶

The Quantopian platform provides data for global equities. Quantopian provides historically accurate equity data (including stocks, ETFs, ADRs, and more) going back as far as 2002 in the US and as far as 2004 in 43 other countries. This includes equities that are no longer trading today.

Asset Identifiers¶

When researching and developing a quantitative investment strategy, it is critical to have a reliable way of identifying equities. However, across the finance industry, there is no one standard that everyone uses to identify equities. Depending on the source of data, equities can be identified using ticker symbols, CUSIPs, SEDOLs, and more. Consolidating data identified using different systems can be a difficult task. To solve this problem, Quantopian collects and surfaces data through uniform APIs by first mapping all datasets to a common set of identifiers called SIDs (security identifiers). SIDs are integer labels that maintain a consistent reference to a particular equity even over symbol changes and other events.

The way that SIDs are determined is different for US and non-US equities. The two methods are described below.

Point-In-Time Data¶

To prevent lookahead bias, Quantopian data is stored in a point-in-time fashion. Each data point is stored with two special fields: an asof_date and a timestamp. The asof_date is typically provided by the data vendor and is used to inform Quantopian's simulation engines about where a data point should be slotted in a timeseries. The timestamp is created by Quantopian upon collecting the data from the vendor and is used to inform the pipeline simulation engine about when in the simulation that data point can be used.

The timestamp of each data point is used to control when pipeline uses that data point in a simulation. Each market has a 'cutoff time' set to 45 minutes before market open. The cutoff time is used to decide if a data point was known early enough to be used by pipeline that day. All data points with a timestamp prior to the cutoff time of day N can be used by a pipeline simulation on day N. Data points with a timestamp after the cutoff time on day N will not be used by a pipeline with a simulation date of N.

Corporate Action Adjustments¶

When your pipeline or algorithm calls for historical data denominated in units per share (such as price per share), it is adjusted for splits, mergers, and dividends as of the current simulation date.

Adjustments depend on three pieces of information:

1. The date of the data point.
2. The date from which the data point is being considered (the current simulation date).
3. Any events (splits, dividends, and mergers) that happened between those two dates.

For example, on June 9, 2014, AAPL had a 7:1 stock split event. If we held one share of AAPL before the split, we would have held 7 AAPL shares after the split. Let's walk through this case.

Let's say our simulation date is May 16, 2014. We want yesterday's close price for AAPL. The date of the price is May 15, and the date from which the price is being considered is May 16. Since no events occurred between May 15 and May 16, we can use the as-is close price ($588.82).

Now let's move forward in time. Our simulation date is July 2, 2014. We want yesterday's close price for AAPL. The date of the price is July 1, and the date from which the price is being considered is July 2. Since no events occurred between July 1 and July 2, we can use the as-is close price ($93.52).

But what if we wanted the close price from May 15, 2014 on this same July simulation date? (For example, if we wanted to run a trailing-window calculation with a two-month lookback period.) Then, we'd see a sudden 84% price drop in the middle of our data window solely due to the 7:1 stock split -- even though there would be no change in the value of your portfolio if you held AAPL.

This will clearly result in misleading values for many graphs and trailing-window calculations (for example, the simple moving average over any window that includes June 9). Fortunately, Quantopian data is adjusted so you don't have to account for these sudden jumps.

To continue with the AAPL example: let's say our simulation date is July 2, 2014; we want the close price from May 15, 2014. Instead of showing a sudden 84% decrease in close price, Quantopian adjusts the pre-June 9 prices so that this sudden jump disappears. Since it's a 7:1 stock split, the May 15, 2014 price will be divided by 7 (adjusted price: $84.12).

In this example, any prices from before June 9 will be adjusted (divided by 7) for simulation dates after June 9, 2014. However, prices will not be adjusted for simulation dates before June 9; and prices from after June 9 will not be adjusted at all.

To summarize: On simulation dates after the split occurs, pricing data from before the split will be adjusted.

Though our AAPL example dealt with a split event specifically, dividends and mergers are dealt with analogously.

If it seems like your data isn't being properly adjusted for a split, merger, or dividend, it's possible that we missed the event. Please reach out to support@quantopian.com if you think this is the case.

Holdout Periods¶

Quantopian provides dozens of datasets including pricing, fundamental, and alternative data. Much of the data is available up to the present day, but some datasets have the last 1-2 years of data held out. Full datasets updated through the current day are available in Quantopian's enterprise offering.