On this page
For a week, the production API of an AI-agent platform I work on as fractional CTO was dying with Worker exceeded memory limit — over and over. A seven-day log census eventually counted 11,058 out-of-memory kills, running at ~66 per hour and peaking at 178 per hour in the hours before we shipped the fix.
Most of them traced back to a single dependency array in a React component.
This is the full story: the mechanism, the wrong turns, the fix, and the rules we wrote down afterwards so we never build this bug again.
Why a Worker runs out of memory
Each Cloudflare Worker isolate gets 128 MB. Two things consume it.
First, a fixed floor. Ours was ~14.7 MB before a single request arrived: an 11.16 MB bundle plus a 3.5 MB Prisma query-compiler WASM module.
Second — and this is the part that reframed everything — ~3.81 MB per concurrent database client. The schema has grown past 180 models, and every request built its own Prisma client, which compiles that schema inside WASM. Two properties make this dangerous:
- The compile is synchronous. It blocks the isolate thread while it runs.
- WASM memory never shrinks.
memory.grow()is one-way. An isolate’s footprint is set by its historical peak concurrency and stays there for life.
Do the arithmetic: 10 concurrent clients × 3.81 MB = 38.1 MB, plus the 14.7 MB floor, and you’re at ~53 MB before any real work happens. A burst that briefly pushes concurrency higher raises the floor permanently.
The tell we almost missed
Before we understood the mechanism, the logs held a clue that didn’t fit any of our theories: an endpoint that queries a 30-row table took 4,992 ms. Twelve of twenty-three “cheap” requests exceeded two seconds — while the database slot queue sat empty at zero.
Cheap endpoints taking five seconds with nothing queued isn’t a slow database. It’s the isolate thread blocked, synchronously compiling schemas for a pile of concurrent clients. Once you know that, the memory numbers and the latency numbers are the same story.
The root cause: one object in a dependency array
One page of the dashboard has a chat. Subscribing to its realtime channel requires an HTTP call to the Worker for channel authorization — and on Workers, that call costs a Prisma client.
The subscription effect depended on an object that was rebuilt inline on every render:
// rebuilt every render → new identity every render
const activeContact =
conversationData?.contact ?? contacts.find(…) ?? null;
useEffect(() => { client.subscribe(channel) },
[activeContact, queryClient, channelName]);
// React compares by identity → teardown + resubscribe, per render
Every render tore down the subscription and re-subscribed, firing a fresh authorization POST. We measured 12 of them in 167 milliseconds, and up to 64 kills in a single second. Not a reconnect loop — a render loop.
Here’s the finding worth framing: those requests return roughly 200 bytes each. They still killed the isolate. The driver is concurrent client construction, not response size. On Workers, a request is not cheap because its response is small.
The other half: one click, twenty requests
The second cause has nothing to do with bugs. Opening a detail view in the app fires about twenty independent requests within a few milliseconds. They land on one isolate, each takes a database client, and the isolate dies partway through.
Worse, the view is a master–detail layout on a single route, so selecting an item is a query-param change, not a navigation — every click repeats the burst. Triaging ten items means ten full bursts. Three of those twenty calls fetched a paginated list just to extract a single value.
What we changed
Four changes shipped together:
- Stabilized the subscription effect. Memoized the object, and made the effect depend on its
userId— a primitive — rather than the object itself. This alone removed the render loop behind ~34% of all kills. - Made channel auth cheaper. The authorization answer was derivable from the session row already loaded, instead of two extra user lookups.
- Stopped redundant refetching. React Query
staleTimeon the projects list went from 0 → 60 s; notifications from 15 s → 5 min. Notification freshness is push-driven anyway — the server publishes an event and the client invalidates, which refetches regardless ofstaleTime. The short window bought no liveness; it just refetched on nearly every navigation. - Lowered the client ceiling from 16 → 10 concurrent Prisma clients. Pure headroom — it caps the worst case around 53 MB.
The result
| Window | Hours | Kills | Per hour |
|---|---|---|---|
| 7-day census, before | 167 | 11,058 | 66.2 |
| Hours immediately before release | 7.1 | 1,264 | 177.9 |
| After release | 18.8 | 18 | 0.96 |
A 99.5% reduction, confirmed the next day. Kills now touch 0.14% of invocations — and that’s an overestimate, because the success-count denominator was itself truncated by an API row cap. The surface that produced thousands of kills went silent, and traffic was healthy throughout, so the quiet is a change in behaviour, not an absence of load.
The house rules we wrote down
The incident is the cheap part. The expensive part is that a Worker is a shared runtime with a hard cap — you cannot add memory to it — and features get built as if it were a server. These are now written rules on the project:
- Count the fan-out, not the endpoint. “Is this endpoint fast?” is the wrong question. The right one is: how many fire at once, and what else fires alongside them? If a page already makes 15 requests on mount, the 16th is not free — it raises the peak concurrency that permanently sets the isolate’s memory floor.
- Never fetch a paginated list to extract one value. Return the derived value from the parent endpoint.
- Frontend fetching decisions are architecture. An unstable
useEffectdependency is a capacity bug, not a rendering nit. Create long-lived clients once in a[]-scoped effect; only the subscription goes in the dependent one. Depend on primitives. - Stream, never buffer.
new Response(r2Object.body)is correct;await r2Object.arrayBuffer()pulls the whole file into a 128 MB budget. Size-gate before reading. - Don’t fix memory by raising a limit. A cap being hit means the work is misshaped. Raising a client ceiling or
--max-old-space-sizeconverts a visible failure into a slower, less obvious one.
The one sentence to remember
On Workers, a request is not cheap because its response is small. Every database-touching request costs memory that is never reclaimed, so the number that matters is how many run at once — which makes burst width an architectural concern that lives as much in the frontend as the backend.
If your serverless backend is dying mysteriously under a healthy-looking load, start with the story of how I found this one — including the three wrong root causes I confidently reported first. That’s its own post.