Reaching for Postgres is the default. It is a good default — most systems have state that has to survive a restart, and reinventing durability is a bad way to spend a month.
But defaults are worth interrogating when the shape of the data is unusual, and market data is unusual. On Finstrm we ended up with no database at all. No Postgres, no Redis, no time-series store. All state lives in process, and the binary rebuilds itself from live feeds on startup.
That sounds reckless. It was not, and the reason why is worth spelling out — because the reasoning generalises even though the answer does not.
The question is not “can we”, it is “what is this data for”
Market data has an unusual property: it is only interesting while it is current. A ticker from four hours ago has no value to somebody asking what the price is now. The system is answering “what is true at this instant”, not “what happened last Tuesday”.
Once you notice that, durability stops being a feature. If the process dies and comes back with an empty store, the correct recovery is not to reload yesterday’s prices from disk — those prices are wrong. The correct recovery is to reconnect to the feeds and rebuild, which takes seconds and produces more correct data than anything you could have persisted.
So the database would have been storing something we would throw away on every restart anyway.
What dropping it actually bought
The performance argument is the obvious one and the least interesting. Yes, reading from a map in memory is faster than a network round trip to Postgres. The measured numbers are on the case study and they are good, but they were never the point.
The real win was operational. Every dependency in a system is a thing that can be down, a thing that needs patching, a thing with its own failure modes you have to learn. Taking the database out removed, in one decision:
- Schema migrations, and the deploy ordering problems that come with them
- Connection pool tuning, and connection exhaustion under load
- Backups, and the restore procedure nobody tests
- A second thing to monitor, alert on and pay for
- An entire category of “is it the app or the database” debugging
The deployment story becomes: copy one static binary onto a machine and run it. There is no second system to stand up first.
When this is the wrong call
This is a narrow pattern, and applying it wrongly is expensive. It works when all of these hold:
- The data is derivable. You can rebuild state from an upstream source. If your system is the system of record, stop reading — you need a database.
- The working set genuinely fits in memory. Not “fits today”. Fits at the scale you expect in two years, with headroom. Ours was under 512 MB for 500 symbols.
- Losing state on restart is acceptable. Measured in seconds of degraded service, not lost customer data.
- You do not need historical queries. The moment somebody asks for a chart of last month, you need storage, and bolting it on later is worse than having planned for it.
Break any one of those and the pattern inverts from elegant to negligent. A user registration system built this way is not clever, it is a data loss incident with good benchmarks.
The general lesson
The useful habit is not “avoid databases”. It is noticing when you are adding infrastructure because it is what one adds, rather than because this system needs it.
Every component you do not add is one you never have to operate, patch, monitor, pay for or explain to whoever inherits the system. That is a real and permanent saving, and it is almost never on the list when the architecture gets drawn.
Ask what the data is for first. Sometimes the answer is that it does not need to outlive the process.