When we started building the Peanut Trading App, we defaulted to REST for everything. It was familiar, well-understood, and easy to debug. Within two weeks, we ripped out half of it and replaced it with WebSockets. Here's what forced that decision — and what we'd do differently from day one.
The Problem with Polling
Commodity prices change by the second. Our initial implementation polled a REST endpoint every 2 seconds to fetch the latest prices. With 500 concurrent users, that's 250 requests per second to a single endpoint, each requiring a database read and JSON serialisation.
The server held up. The user experience didn't. Two-second polling means a user might see a price that's already 1.9 seconds stale — in a trading context, that's the difference between a profitable order and a missed opportunity.
We needed sub-second updates. REST polling can't deliver that without destroying the server.
How WebSockets Work (Simply)
REST is request-response: the client asks, the server answers, the connection closes. Every request carries full HTTP headers — typically 500–800 bytes of overhead, even if the payload is 20 bytes of price data.
WebSockets establish a persistent connection between client and server. Once open, either side can push data to the other at any time — no request overhead, no repeated handshakes. A price update becomes a tiny JSON frame pushed instantly to every connected client.
What We Built
Our WebSocket server maintained a single shared price feed (sourced from an upstream commodity data API). When a new price arrived, it broadcast to all connected clients in a single pass — O(n) with minimal overhead per client.
Connection state was the hard part. WebSocket connections drop — bad signal, app backgrounding, server restarts. We implemented:
- Client-side automatic reconnection with exponential backoff
- A
lastPricecache so reconnecting clients get the current price immediately - Heartbeat ping/pong every 30 seconds to detect silently dead connections
- Server-side connection registry to clean up stale connections
What Broke First
Under load testing with 500 simulated connections, two things failed:
File descriptor limits. Each WebSocket connection holds an open file descriptor. Linux defaults to 1,024 per process. We hit this ceiling immediately and got connection refused errors. Fix: raise the limit in system config (ulimit -n 65536) and configure the same in Docker.
Goroutine leaks. Our Go server spun up a goroutine per connection for reading. When connections dropped without a clean close, the goroutines leaked. Fix: context cancellation tied to connection lifecycle, with a timeout on reads.
When to Use Each
After building both sides of this, the decision is simple:
- Use REST for: creating orders, fetching account data, any request-response interaction where the client drives the timing
- Use WebSockets for: live price feeds, notifications, chat, anything where the server needs to push to the client unprompted
WebSockets aren't better than REST. They solve a different problem. Use the right tool for the direction data flows.
In Peanut, we ended up with a hybrid: WebSockets for price feeds and order status updates, REST for everything else. That's the architecture most real trading platforms use — for good reason.