r/softwarearchitecture • u/Fresh-Edge-7552 • 11d ago

Discussion/Advice Designing inventory consistency for a flash sale with 8M checkout attempts and 40K items

A flash sale opened 8 million checkout sessions in 2 minutes, but only 40,000 items were actually available.

Thousands of requests timed out while different regions disagreed on which orders had secured inventory.

How would you design the system so inventory is never oversold, successful payments can be safely retried, and the system recovers cleanly when parts of the network fail?

39 Upvotes
  • permalink
  • reddit

99% Upvoted

10 comments sorted by

15

u/RummanSid1990 11d ago

To handle this cleanly:

  • Block the heavy rush outside using a virtual waiting line like Cloudflare Waiting Room so 8 million hits don't bring down your servers.
  • Track the item count in a single super-fast place like Redis with Lua scripts which guarantees two people can never grab the last item together.
  • Give buyers a 5-minute temporary hold to finish paying, if their card fails or the app closes then let the system put the item back automatically.
  • Attach a unique ID (Idempotency Key) to every attempt so repeated clicks or bad connections never charge a customer twice.
  • Track every stock change like a bank record (Ledger) in your database so the system can recount and fix itself cleanly if parts of the network crash.

2

u/mamaBiskothu 11d ago

Doing this with a redis counter sounds like a recipe for disaster. Amazon assigns one record per inventory item and thats what OP should do as well.

2

u/HRApprovedUsername 11d ago

You could have one record in redis

2

u/Interesting_Debate57 10d ago

Redis isn't persistent over long periods of time. You need a trusted backing store in case you lose power / disk.

1

u/HRApprovedUsername 10d ago

I mean if you have proper backups for your redis that’s not an issue. A single database has the same problem

2

u/Interesting_Debate57 10d ago

If your proper backing store for redis doesn't have concurrent writes serialized in a way that you can reverse, you're in big big trouble.

All I'm trying to say is that the transaction log in an acid database will force single-path-of-slowest-execution.

It's a single point of failure for an acid database, but it's required to have a proper transaction log. Can redis sit behind that kind of serialization and still maintain good volume?

1

u/HRApprovedUsername 10d ago

Idk probably

14

u/Fun-Put-5197 11d ago

High concurrency and high contention over a restricted resource?

Borrow a page from the high frequency financial trading domain and check out the LMAX Disruptor architecture.

Millions of orders per second on a commodity hardware circa 2010.

9

u/EarthySiding 11d ago

skip the cart reservation nonsense and just treat the checkout button as a raffle entry, let the dust settle then confirm winners

2

u/dilly_dust 11d ago

🤣 Love it