Four sources, in the order you should reach for them. None of them cost money. The last one is the one you will end up relying on.
1. Public tick archives
Full-depth order books for the short crypto markets exist on Hugging Face and GitHub, some at 100-millisecond resolution, covering weeks at a time and already labelled with official outcomes. Search the dataset hubs for the exchange name plus "crypto" or "up down". Check the span and the sampling rate before you build on one; two archives with the same name can be a month apart.
2. Exchange endpoints
Kalshi serves candlesticks for any market back to listing: bid, ask and volume at the interval you ask for. Polymarket serves a price history per token and every settled market by numeric id, including markets that no longer appear in slug queries. Both are free order-book history with no logger required.
3. On-chain fills
Every Polymarket fill is an event log on Polygon; Limitless fills are on Base. An indexer pulls a wallet's whole trading history in minutes, which is how the copy-trading case study was tested and how anyone can audit a public win rate.
4. Your own logger
Websocket books plus a 5-second snapshot, compressed daily. Expect 1 to 2 GB a day per exchange. Start it before you start anything else; the week you spend building a model is a week of data you will wish you had.
How to backfill
- Fetch the official result for every market by id. Never derive it from the last price or from your own oracle read; those were wrong 2 to 7 percent of the time in the logs, and one 14 percent "edge" turned out to be nothing but the wrong labels.
- Keep the raw payloads and re-derive features later, never the other way round.
- Compress daily with zstd; a month of ticks fits on a small NVMe.
- Record the clock every book was read on. A window without a fresh read is not a trade you could have made.
- Drop windows whose book never changed from the open to your entry second. About one window in eight in one exchange's export was a frozen quote from an outage, and a naive replay "won" 92 percent of them.
Spot prices, four options
- Exchange websocket
- Binance or Coinbase book ticker. Cheapest and fastest; the prediction-market book reacts to it in about 16 milliseconds, so it is a reference, not an edge.
- 1-second klines
- Binance publishes daily archives. Enough for research, not for firing orders.
- The exchange's oracle stream
- Polymarket relays its Chainlink prices over its own websocket. The only feed that matches settlement.
- Paid data streams
- Sub-second oracle feeds cost more than most bots earn. Measure with the free feeds first.