TickStream
Merging live crypto order books from multiple exchanges into one real-time view
TickStream is a consolidated crypto order book written in Go. It ingests live feeds from Coinbase and Kraken, reconstructs each exchange's order book correctly from snapshots and updates, and merges them into a single unified view. This allows traders to see the best available prices across connected exchanges in real-time.
The problem
For cryptocurrency traders, having a unified view of order books from multiple exchanges is essential for making informed decisions. However, each exchange sends data in its own format, messages arrive out of order, and gaps can occur due to network issues. Merging these feeds into a consistent, real-time view without errors or delays is technically challenging, as it requires handling data integrity, synchronization, and concurrency without blocking readers.
What I built
TickStream is built and tested, with measured end-to-end apply latency of 7.4 µs at p50 and a wait-free read path at 41 ns. It has been verified with concurrent assertion tests for immutability, checksum tests, and a full apply-semantics matrix. The system is open-sourced on GitHub, demonstrating a working implementation of a consolidated order book in Go.
How it works
TickStream uses a single writer engine that processes order-book updates from each exchange in a dedicated goroutine. Each feed decodes and normalizes messages, sending them to the engine via a bounded channel. The engine applies updates to per-venue books, verifies integrity using checksums (like CRC32 for Kraken), and resyncs from fresh snapshots on failures. It consolidates the books into a single best bid and offer. The system publishes a new immutable snapshot after each update, allowing any number of readers to access the data wait-free via an atomic pointer load. This design avoids locks and ensures readers never block the writer. Key challenges included ensuring snapshot immutability (not reusing memory across publishes), handling full channels without blocking (dropping frames and resyncing), and maintaining correctness during high-throughput scenarios.
Highlights
- Wait-free read path using atomic pointer loads, with p50 latency of 41 ns
- Immutable snapshot publication ensures readers never see torn data
- Bounded channel hand-off drops frames on full buffer to prevent writer blocking
- Checksum verification (CRC32 for Kraken) triggers full resync on mismatch
- Concurrent assertion tests enforce snapshot immutability invariant
- Measured end-to-end apply latency with coordinated-omission-aware benchmarks
Let's build yours.
Tell me what you're trying to ship — you'll get a scoped plan and a straight answer.