The Forum Was the Easy Part
I wanted a private place for the training community I write for. Somewhere people could post, argue, share what they are working on, and find it again a month later. Not a Discord that scrolls into the void. Not a subreddit run on someone else’s rules and someone else’s ranking. Not a Facebook group. A real forum, invite-only, that I own.
There was a colder reason too. One AI crawler hit my personal blog with over three hundred thousand requests in a single day, trying to swallow everything I had ever written. That is not a typo, and it was one crawler on one day. The open web is being strip-mined for training data, and I did not want the community’s conversations to be the next thing fed into a model without anyone asking. A login wall in front of the whole thing settles it. What members write stays theirs.
So I built a forum for people to post training-related musings.
It works. People post, reply, react, get notified, search, upload images, and read it on their phones. But “it works” is not the same as “it runs well on a platform that bills by the row,” and the second is where the engineering starts.
I host the whole thing on Cloudflare’s edge. The API is a Worker, the frontend is on Pages, the database is D1 (their serverless SQLite), images sit in R2, and Cloudflare Access handles login. It is fast for the people using it, it fits inside the free tier, and at this size it costs nothing. Keeping it that way while the forum gets used is the part worth writing about.
Why I Wrote My Own Forum
The first fair question is why I wrote a forum at all. Good forum software exists. Discourse is good. There were two reasons.
The practical one is that the off-the-shelf options want a real server. Discourse alone expects Postgres, Redis, and a couple of gigabytes of memory sitting there whether anyone is posting or not. That is a machine to run, patch, and pay for, which is the exact opposite of what I wanted. The whole appeal was a thing that costs nothing while it sits idle and that I never have to log into a server to repair.
The other is that I wanted to own every parameter. When you install someone else’s forum, you inherit their data model, their queries, and their performance characteristics, and you optimize in the cracks they leave you. When you write your own, every knob is yours. That is more work, and it is also the only reason the rest of this article exists. I could change how a count is stored because the count was mine to change. None of the optimization below is reachable on a platform you rent by the month and tune through a plugin settings page.
So I built it myself, on Cloudflare’s edge, for reasons I will come back to once the constraints that choice imposes have shown their teeth.
What Got Built
Everything after this is about making the forum cheap and correct to run, and it is easy to lose sight of how much forum there is to run. Before the hardening, here is the surface area it sits on, in one table. The rest of the article picks a handful of these and digs in.
| Area | What is in it |
|---|---|
| Structure | Categories hold threads which hold posts. Per-category visibility and access (hidden, read-only, full), with per-group and per-user overrides. |
| Writing | A Markdown editor with live preview, posts up to forty thousand characters, quoting that handles whole posts, partial selections, several paragraphs, and nested quotes that collapse, an arrow-coercion shortcut, and a formatting cheat sheet. |
| Drafts | Autosaved per-thread drafts, title included, up to three kept at once behind a picker, with blank ones ignored so they do not eat a slot. |
| Reading | Unread tracking that drops you at your first unread post and marks where new replies begin, watching and a Watching page, relative timestamps with the exact time on hover, First and Last page jumps, scroll-position restore on back and forward, and a back-to-top button. |
| Reacting | Per-post reactions with notification pile-ons collapsed into one entry, polls with editable votes, a private saved-posts list, and bookmarks. |
| Media | Image upload to R2 (magic-byte checked, EXIF stripped, served from a category-private proxy), external image embeds, a click-to-zoom lightbox, lazy loading, and horizontal scroll for wide tables. |
| Messaging | Private one-to-one direct messages with Markdown, an unread count, and mutual blocking. The privacy policy says plainly that messages live in the database and an administrator with direct access could read them. |
| Notifications | In-app notifications for replies, mentions, reactions, and staff actions, plus an opt-in daily or weekly email digest, coalesced per thread, with an unsubscribe link that works without logging in. |
| Mentions | An @mention autocomplete backed by the cap-and-handoff roster from later in this article. |
| Moderation | A report-a-post queue with a count badge, soft delete with a trash and restore, post edit history (the last ten versions), thread lock, resolve, and reopen, and per-thread notification mute. |
| Admin | Category, group, and user management, a media manager, settings, a moderation dashboard, a send-test-digest button, and a server-side demo mode that swaps every real post for a stand-in so screenshots leak nothing. |
| Accounts and legal | Identity through Cloudflare Access, roles for member, moderator, and admin, versioned rules and privacy policy with re-acceptance and a “what changed” summary, and full account anonymization. |
| Reliability | Per-user daily write ceilings, Sentry error reporting, a health endpoint, weekly encrypted off-site backups with a monthly restore check, friendly error pages, and a front-end test suite. |
The Rows-Read Budget
D1’s free tier gives you five million rows read per day, and a Worker on the free plan gets fifty subrequests per invocation. This means the priority is optimizing those two numbers, not latency or CPU time. More specifically, rows read.
That one fact shaped more of this project than any framework choice. A query that returns a single row but reads a million still costs a million against the daily budget. So the question I started asking about every hot query was not “is this fast?” It was “how many rows does this touch to do its job?”
The Worst Query in the App
Every thread page shows each author’s lifetime post count next to their name. The member directory shows it too. That count was computed at read time, with a COUNT(*) over the author’s entire post history, on every page load.
On the latency dashboard, this query was nothing. About one percent of runtime, sub-millisecond at the median. If I had only been watching latency, I would never have found it.
On the rows-read dashboard, it was the single heaviest pattern in the application. Ninety rows read for every row returned. The member directory was worse, because it ran a correlated subquery per member and counted each one’s whole history on every page of the list.
The obvious fix is a better index. The obvious fix is wrong. There is no index that makes counting cheap. Counting is reading. To produce the number “412 posts,” the database has to walk 412 rows, and an index only changes which 412 rows it walks. Batching the per-member subqueries into one grouped query felt like progress, but it reads the identical rows. It just reads them in one statement instead of many.
The only way to stop reading a thousand rows to produce one number is to stop counting at read time.
So I stored the count. One migration adds two columns to the users table and backfills them from the real data in a single statement:
ALTER TABLE users ADD COLUMN post_count INTEGER NOT NULL DEFAULT 0;
ALTER TABLE users ADD COLUMN thread_count INTEGER NOT NULL DEFAULT 0;
UPDATE users SET
post_count = (SELECT COUNT(*) FROM posts p JOIN threads t ON t.id = p.thread_id
WHERE p.author_id = users.id AND p.deleted_at IS NULL AND t.deleted_at IS NULL),
thread_count = (SELECT COUNT(*) FROM threads t
WHERE t.author_id = users.id AND t.deleted_at IS NULL);Look at the join in the post_count subquery. A post counts only if its own row is live and its thread is live. A reply sitting in a soft-deleted thread reads as not there, which is exactly what the old read-time COUNT(*) produced. Getting that predicate identical to the query I was replacing is the whole reason the backfilled numbers came out right. The tables were altered in place over time. If this ever becomes a clonable, deployable repo, the schema will ship with these columns built in.
The read sites turned into a column read. The thread page already joins the users table, so it gets the number for free. The directory dropped its subquery entirely. The interesting part of denormalizing the counter was not the counter. It was keeping the count honest.
I did not decrement on delete. The maintenance splits by how risky the path is. The hot path, posting, takes a cheap increment. The rare paths, delete and restore, recompute from the actual rows.
A new thread gives its author one post (the opener) and one thread, unconditionally, because that increment rides in the same transaction as the opener insert and commits only if the thread does:
A reply is trickier, because the reply insert is itself guarded. You can only reply to an open thread in an active category, and if that guard rejects the insert, the count must not move. So the increment re-encodes the same guard as an EXISTS and fires only if a real reply could have landed:
UPDATE users SET post_count = post_count + 1
WHERE id = ?1
AND EXISTS (SELECT 1 FROM threads t JOIN categories c ON c.id = t.category_id
WHERE t.id = ?2 AND t.status = 'open'
AND t.deleted_at IS NULL AND c.visibility = 'active')That EXISTS is the kind of correctness you only reach because the count is yours. A plugin hook firing on “reply submitted” would have drifted every time the insert underneath it was a no-op, and nobody would have noticed until the numbers were visibly wrong.
On delete or restore I do not adjust the number at all. I recompute it from canonical rows:
UPDATE users SET
post_count = (SELECT COUNT(*) FROM posts p JOIN threads t ON t.id = p.thread_id
WHERE p.author_id = users.id AND p.deleted_at IS NULL AND t.deleted_at IS NULL),
thread_count = (SELECT COUNT(*) FROM threads t
WHERE t.author_id = users.id AND t.deleted_at IS NULL)
WHERE id IN (/* the affected author ids */)Recompute cannot drift. It also sidesteps every “but was the parent thread also deleted?” edge case automatically, because it just asks the database what is true right now.
There was one case that took real care. Soft-deleting a thread does not stamp its replies as deleted, because a restore has to bring the whole thread back intact. But while the thread is gone, those replies have to stop counting toward their authors’ totals. A thread is not one author’s content. So the delete and restore path recomputes for every distinct author who has a post in the thread, not just the person who started it:
UPDATE users SET /* the same recompute as above */
WHERE id IN (SELECT DISTINCT author_id FROM posts WHERE thread_id = ?1)Recompute for only the thread’s author and a deleted thread leaves every replier’s count one too high, with nothing to flag it, because the number still looks plausible. Rare in practice, and still worth handling.
Before I let anything read the new columns, I ran the backfill and then checked it against the truth it came from. The check is just the backfill’s own expressions in a not-equal comparison:
SELECT COUNT(*) AS mismatches FROM users u
WHERE u.post_count <> (SELECT COUNT(*) FROM posts p JOIN threads t ON t.id = p.thread_id
WHERE p.author_id = u.id AND p.deleted_at IS NULL AND t.deleted_at IS NULL)
OR u.thread_count <> (SELECT COUNT(*) FROM threads t
WHERE t.author_id = u.id AND t.deleted_at IS NULL);It had to return zero before the reader shipped. It did.
The heaviest rows-read pattern in the app is now a column read.
Putting numbers on it: at the forum’s traffic, on a working estimate of a thousand thread views and eighty directory loads a day, those unchanging counts were reading on the order of two million rows a day. Call it forty percent of the five-million-row budget, spent reprinting numbers nobody had touched. Denormalizing them took that to near zero. Every other change in this article saved a few percent at most.
The shape of the win, before and after:
Before: read-time COUNT |
After: stored counter | |
|---|---|---|
| To show one author’s total | walk the author’s whole post history | read one column off the users join |
| Rows read per row returned | about 90 | 1 |
| Footprint on the runtime dashboard | about 1 percent, invisible | negligible |
| Share of the 5M daily rows-read budget | roughly 40 percent | near zero |
Paying for Two Trips When One Would Do
Almost every list in the forum is paginated, and almost every paginated endpoint did the same thing. It awaited the total-count query. Then it awaited the page-of-rows query. Two database round-trips, one after the other, with the second query having no dependence on the first.
On the edge, every database call is a network hop. Two sequential awaits that do not depend on each other are two trips for one logical step, and the request waits for the sum of both instead of the longer of the two. So I fired them together.
const [totalRow, rows] = await Promise.all([
c.env.DB.prepare(countSql).bind(...).first<{ n: number }>(),
c.env.DB.prepare(pageSql).bind(...).all<RowType>(),
]);That went across every paginated route. Recent, watching, category threads, the member directory, profiles, bookmarks, notifications, the admin tables. Search was the best case, because its total, its body-text matches, and its title matches were three independent queries run one after another. They now fire together, and three trips became one.
I used Promise.all rather than D1’s batch, and the reason is small. The count and the page return different row shapes, and Promise.all keeps each one’s types intact while batch hands you back a uniform array you then have to index and re-cast. The forum is nowhere near the fifty-subrequest limit, so the extra subrequest costs nothing. If I were subrequest-bound, the answer would flip. That is not to say I never will be, but the design is aimed at a relatively small, private user base.
The thread lists also show an unread dot. That dot came from a third query, run after the page loaded, that looked up the reader’s position in each thread on the page. It depends on the page results, so it cannot join the parallel batch. I assumed that made it a permanent third trip.
It does not. “Needs the rows” only rules out running it in parallel. It does not rule out doing the work inside the page query itself. The unread dot is just a comparison between a thread’s latest post and where the reader left off, which is a join and a computed column:
SELECT t.*, u.display_name AS author_name, c.name AS category_name,
(t.last_post_id IS NOT NULL
AND t.last_post_id > COALESCE(tr.last_read_post_id, 0)) AS unread
FROM threads t
JOIN users u ON u.id = t.author_id
JOIN categories c ON c.id = t.category_id
LEFT JOIN thread_reads tr ON tr.thread_id = t.id AND tr.user_id = :uid
WHERE c.visibility = 'active' AND t.deleted_at IS NULL
ORDER BY t.last_post_at DESC
LIMIT :n OFFSET :offThe LEFT JOIN is doing the work. A thread the reader has opened has a matching thread_reads row, so the comparison against last_read_post_id resolves normally. A thread they have never opened has no row at all, COALESCE supplies a zero, and any thread that has a post comes back unread. One query, correct for both cases, and it rides the thread_reads primary key on (user_id, thread_id) so the join costs nothing.
That is the exact comparison the unread-count badge already ran, so I knew the shape worked. The three busiest thread lists went from three trips to one, and two helper functions that existed only to run that third query got deleted.
If a second query only reads something the first already returned, fold it into the first query’s SQL. Keep a separate query only when the code has to do something with the rows that SQL cannot express.
Caching: Free Until It Guards a Door
The category list gets read on nearly every request, because deciding what a user can see starts with knowing what categories exist. It is a tiny table that changes when an admin adds or edits a category, which is to say almost never. It was being read from the database on every request anyway.
So I cached it in the Worker, with a ten-second lifetime and an explicit clear whenever a category is created, edited, or deleted. About a tenth of the database runtime, gone, for a cache and a clear-on-write.
That win shows up in runtime, and the unit matters here. The category table is small enough that reading it barely registers against the rows-read budget. What caching it saves is a round-trip and the CPU to run the query, on nearly every request. This is the rare fix that moves the latency and subrequest meters and leaves the rows-read one alone.
There is a catch.
Reading the category list to decide what a user can see is safe to cache. A ten-second-stale view of which categories exist hurts nobody. But the same data also decides whether a user is allowed to post somewhere, and caching that means a write-permission decision can be up to ten seconds stale. That is a different kind of risk than a stale read.
I made that call deliberately, at a quiet hour, knowing the write path re-checks the category in the insert itself as a backstop. Caching a read is free. Caching anything that guards a write is a decision and I feel that this was the write one (get it?).
The Download Budget
Every optimization so far was about the database. There is a second budget the database never sees: how much code the browser has to download before the page is usable.
A forum has a lot of screens most people never open. The admin console, the moderation queue, the media manager, the group editor. And it has one screen that is heavy for everyone and used only in bursts, the Markdown editor, which drags in a parser and its linkify and highlighting pieces. Bundled together, all of that is one download that every visitor pays on first load, including the member who only ever reads.
So none of it loads until it is asked for. Every route is a lazy import behind a loading boundary, and the editor is split out the same way. Open the forum and you get the shell, the thread list, and the reader. The editor arrives the first time you go to write something. The admin code never arrives at all unless you are an admin who opens an admin page. The initial download came down by about forty-five percent.
It is the rows-read lesson aimed at a different meter. What costs you is the size of what you need in order to do the thing you came for. A reader should not have to download the editor, and nobody but an admin should ever download the admin panel.
Holding the Phone
Profiling finds what is slow. It did not find any of what was wrong with the forum on a phone.
The first thing it caught was the editor. I went to type a reply on my phone one evening and the thing could not keep up with my thumbs. Letters landed a beat late and the whole box stuttered as I typed. On my desktop it had always felt fine, so I had never once noticed.
Two things were firing on every keystroke. The Markdown preview was re-rendering the whole post even while I sat on the Write tab, where the preview is not on screen at all. And the draft autosave was writing the entire body to local storage one character at a time. On a desktop you cannot feel that. On a phone you feel every bit of it.
The fix was to stop doing work nobody asked for. The preview only renders when you are looking at the Preview tab. The autosave waits until you stop, debounced to six hundred milliseconds, instead of running on every letter. Neither change can lose a draft, because the save still flushes when you leave the page, switch tabs, or send the app to the background, and submitting cancels any pending save so a posted draft cannot come back from the dead. Typing on a phone went back to feeling like typing.
As a side note, I am not happy with Substack and whatever degree of autosaving sans debouncing is going on over there. It is legitimately painful to type on that platform. :(
The mobile navigation was a single row that scrolled sideways, and the scrollbar was hidden. So you had to swipe through eight to sixteen destinations, and nothing on screen told you there was anything past the right edge. It was the kind of thing that is invisible in a screenshot and obvious in your hand.
I replaced it with a normal slide-out drawer behind a menu button. Everything stacked vertically, nothing to swipe, room for every link a moderator or admin sees.
The one part that fought me was a layout trap I had to learn by breaking it. The header slides out of the way as you scroll, which it does with a CSS transform. An element with a transform becomes the anchor for any fixed-position element inside it. So when I put the drawer inside the header, the drawer anchored itself to the header’s little box instead of the whole screen. Moving the drawer out of the header, as a sibling, fixed it completely.
The desktop version had a quieter version of the same crowding. An admin with every staff link visible saw sixteen items wrap onto three rows. I collapsed the moderation and admin links into one dropdown, so the bar went back to one clean row, and I put the open-reports count on the dropdown itself so urgent moderation work does not hide behind a click.
The @Mention Roster
The forum has a mention feature. Type an at-sign, get a list of members to autocomplete against. To do that, the editor fetched every member once when you opened it, kept the list, and filtered it locally as you typed.
The reflex when you read “it fetches every member” is to want to fix it. I had to argue myself out of that.
Fetching everyone and filtering in the browser is the correct design at this size. The list is a few kilobytes, the matches appear instantly, and it costs zero requests per keystroke. The alternative, asking the server on every keystroke, is slower for the person typing. Building it now would make the feature worse to use, in exchange for solving a problem the forum does not have.
You also cannot just paginate it. The member directory is paginated already. But this is a typeahead, and a typeahead is not a list you scroll. You either hold all the candidates or you search on the server. There is no third option.
So I made the simple version aware of its own ceiling. The full-roster fetch asks for one row more than the cap of a thousand, which makes the “is this everybody?” flag exact and costs nothing extra:
If that thousand-and-first row comes back, there is at least one more member than the client should try to hold, so the response is marked complete: false and trimmed to a thousand. Under the cap, the client filters locally exactly as before, instant and free. Over the cap, the flag flips and the client switches to asking the server as you type, debounced. The server side is a prefix match written as a range scan so it rides the existing unique index on the lowercased name instead of scanning the table:
SELECT id, display_name FROM users
WHERE display_name_lower >= ?1 AND display_name_lower < ?2
ORDER BY display_name_lower LIMIT 10The two bound values are the typed prefix and that same prefix with U+FFFF appended, the highest character in Unicode’s basic plane, so a query for al matches every name from al up to but not including that upper bound. A LIKE 'al%' would have meant a scan. The range bound turns the same question into an index seek.
The handoff is automatic. There is no setting to change later. The payload and the rows read stay bounded no matter how large the forum gets, and I paid nothing in user experience to get there, because the better-feeling version stays in charge right up until it cannot.
Do not pre-build the scale solution when the simple one is both easier and nicer below the threshold. Teach the simple one to notice the threshold and hand off.
Why Cloudflare
I have spent most of this article fighting a platform’s constraints. It is fair to ask why I chose a platform whose constraints were going to be the work.
I did not pick Cloudflare because it is trendy. I picked it because I did not want to think about it after it was running. Plus I have used Cloudflare across various fire-and-forget projects and have never had any issues.
The forum is a side project, and the thing I am least willing to spend on a side project is operations. A virtual server means I am now in the business of patching it, watching it, and being the reason it is down at 2am. A managed database plus a separate frontend host means two bills and two dashboards to keep in sync. Both are fine. Neither is what I wanted to be doing.
Cloudflare let me run the whole thing without operating anything. The Worker, the static site, the database, the image storage, and the login wall are all one platform, on one dashboard, billed as one thing. At this size that one thing is zero dollars, and it stays near zero as the forum grows, because the pricing is per use and the use is small.
There is a quieter reason, and it is about trust. I have used AWS. AWS has a way of growing fees you did not ask for. A monitoring line item here, a thing that bills per request there, and when you finally want to turn it off you spend an afternoon learning which of a dozen consoles owns the switch. I did not want a platform I had to defend myself against. Cloudflare’s free tier has hard, legible limits, and when I reach one it tells me instead of quietly charging me for it. GCP is marginally better than AWS and we don’t talk about Azure…
That last part connects back to everything above. A platform that charges by the row, and counts every one, is the same platform that will never surprise me with a bill. The five-million-row ceiling that sent me hunting for a hidden COUNT is the same reason I have never once logged in worried about what last month cost. A ceiling I can see is a thing I can design against.
The domain, the DNS, and the login layer were already here too. So the real comparison was never “Cloudflare versus AWS.” It was “use what is already under my domain and costs nothing” against “go stand up a server I will have to babysit.” Put that way, it was not a hard call.
I gave things up for it. D1 is SQLite under the hood, so I do without the richer Postgres feature set. The Workers runtime runs in short-lived isolates. There is no long-running server underneath, so no process memory that outlives a request and no connection pool to keep warm. Those constraints are real. They are also, as the last several sections show, where nearly all the interesting decisions came from. A platform with no hard edges would have let me ship the same forum without ever once having to think this hard.
Operations
I said I did not want to operate anything. That was the goal. A forum with real members still has to be backed up, watched, and reachable, and none of that comes free just because the platform is managed. So I built each piece once and made it run without me in the loop.
Start with the data, because it is the one thing the platform does not hand back to me on its own. D1 is durable, but that only covers the platform losing my data, not me destroying it myself. Once a week the whole thing is exported, encrypted, and pushed off-site. That part is ordinary. The part I care about is that once a month a job pulls the latest backup, restores it into a throwaway database, and checks that it actually came back. I would rather learn it is broken on a quiet Tuesday, in a scratch copy, than on the one day I need it for real.
Deletion is staged, for a reason close to why the backups exist. Nothing a member removes is destroyed on the spot. A delete hides the content at once but keeps the row for thirty days, and only then does the nightly maintenance job purge it for good. Someone who deletes a post in a bad moment has a month to undo it, and after that it is genuinely gone, which is what the privacy policy promises. The same nightly job reclaims any uploaded file that never got attached to a post, so storage does not slowly fill with abandoned images.
For knowing when something breaks, the app reports itself. The Worker is wrapped so that any unexpected error in a request, or in the nightly cron, is captured and sent to Sentry with a stack trace, and the browser app does the same for crashes in someone’s tab. I hear about a bug from a dashboard now, not from a member telling me the site is acting weird. Two details I am quietly pleased with: with no DSN configured the whole thing compiles down to a no-op, so the code shipped before the monitoring existed, and it never sends personally identifying data, because the Worker handles members’ writing and that is none of Sentry’s business. That Sentry ingest host is also the one outside origin the content policy lets the browser reach, which is the entire reason connect-src in the content policy later in this article is not just 'self'.
The front door has its own check, and it is deliberately dumb. A health endpoint answers { ok: true } and nothing else, so an uptime monitor can ping it on a schedule and confirm the Worker is alive without ever touching real data.
Email runs the same way. The digest goes out through Resend on a scheduled cron, in small hourly batches sized to stay under the free plan’s send limits, and the one part with teeth is the signed bounce webhook from the security section. A hard bounce or a spam complaint comes back signed, the app checks the signature, and that address quietly stops getting mail. I do not run a mail server, I do not chase bounces by hand, and one bad address cannot drag the rest of my sending down with it.
The first time one of those notifications pulled someone who was not me back into a thread, the forum quietly stopped being a side project and became a place. That is the only metric in this whole article I did not profile.
None of this is much code. It is the smallest amount of operating that lets a side project hold real people’s writing without turning into a second job.
The Tests, and the One That Fought Back
I over-test by temperament. A numerics library I wrote has a suite that borders on paranoid, and the forum inherited the reflex. The denormalized counters, the access rules, the CSRF guard, and the search tokenizer all have tests whose only job is to fail loudly the day I break them by accident.
Two are worth naming, because each guards against a specific future mistake rather than a present one. The first reads the live users schema and forces every column to be classified as either scrubbed or kept when an account is anonymized. A later migration cannot quietly add a column of personal data that the privacy scrubber then forgets about, because the test goes red the moment a column shows up unclassified. The second drives the counter maintenance through its worst path: create, a single reply deleted and restored, and then the whole-thread delete and restore where several authors lose and regain a post at once. That multi-author case is the one I was most likely to get wrong, so it is the one with the most tests pointed at it.
Then there was the test failure that was not a bug.
The API suite runs on Cloudflare’s vitest-pool-workers, which stands up the real Worker runtime for each test file. On Windows, the full suite started failing at random. A different module would fail to load every run, there were zero assertion failures, and every test that did load passed. It looked exactly like flaky code.
It was not the code. The pool does not bundle dependencies. It lazy-fetches every unbundled module over a loopback socket, and with all the test files sharing one host process those connections pile up until Windows starts refusing them. The failure was the harness, not the forum. The fix was a runner that spawns one fresh process per test file, so the sockets are released between files. If I had assumed the red was mine, I would have spent a day hunting a bug that did not exist. It cost five minutes the second time because I wrote down, in plain words, how to tell this particular red apart from a real one.
Where It Breaks Next
The roster handoff future-proofed one feature. I should be clear about where the rest of the system runs out of room, because pretending it scales forever is the opposite of the point.
It helps to say what this was built to be. It was always meant to be a small, invite-only community for people who mostly already know each other. The design was never pointed at a hundred thousand strangers, so the walls below sit a long way out, and they sit there on purpose. Designing for more would cost money and buy nothing the community needs.
The most concrete ceiling has nothing to do with rows read. It is the front door. The whole forum sits behind Cloudflare Access, which is free up to fifty seats. The fifty-first member is where that ends, and the pricing does not ease you across it. Crossing fifty moves the entire account onto the paid tier at seven dollars per user per month, billed for everyone at once, so the day a fifty-first person joins, the login layer jumps from zero to roughly three hundred and fifty dollars a month. At the size this was built for, that is free. Anywhere past it, the auth layer becomes the first bill I would ever have to think about, which is its own argument for keeping the room small.
The other walls are about load rather than headcount, and they sit further out. I will take them in rough order of how soon they would matter.
Closest is the rows-read budget itself. Everything above bought real headroom against it, but headroom is not infinity. If the forum gets popular enough, five million rows a day is a wall, and no amount of cleverness gets past it. You pay for more. That is the cheapest wall to hit, because it is a number on a billing page and not a rewrite.
Behind it is notification fan-out, which leans on the scarcer budget. Reads get five million rows a day. Writes get only a hundred thousand. When someone replies to a thread, everyone watching that thread gets a notification row written. A normal thread is nothing. A thread that four hundred people are watching writes four hundred rows on a single reply, which is four tenths of a percent of the entire daily write budget spent on one post. A few hundred replies like that in a day would spend all of it. For a community this size that is a comfortable margin, and it is still the first thing I would move to a queue, or to a fan-out-on-read query, if it ever stopped being comfortable.
Then search. D1’s build of SQLite full-text search does not include the function that builds result snippets, so I build them in code, which means search pulls up to a thousand candidate rows per query and trims them in the Worker. It is fast and bounded now. It is the kind of thing that stops being fast when the archive is ten times larger.
Last is the cache. The category cache lives inside one Worker isolate, so it helps within that isolate’s short life and does nothing across isolates. Every cold isolate still pays one database read to warm up. If cache hit rate ever becomes the bottleneck, the answer is a shared cache layer, which is real work I have not needed to do.
None of these are on fire. I name them so that when one starts smoking, I am not surprised by where it came from.
And there is a cheap valve under all of it. If a wall ever did get close, getting past it means flipping to the paid plan and nothing more. That plan includes twenty-five billion reads a month, and anything beyond it runs a tenth of a cent per million rows read and a dollar per million written. For a community this small, that is the difference between paying nothing and paying for lunch. The whole design was meant to keep the forum on the free tier for as long as it stays small, and if it ever outgrows that, the cost is a few dollars a month and no rewrite at all.
The Security Posture
Most of the security work happened before this hardening pass, which is the right time to do it. I will state the posture plainly rather than dress it up.
Login is not my problem, on purpose. Cloudflare Access sits in front of everything and proves who you are before a request ever reaches my code. I do not store passwords. I do not run a session system. What reaches the Worker is a signed JWT from Access, and the Worker does not take it on faith. It verifies the signature against Access’s published keys, checks that the issuer and the audience tag match my application, and pins the algorithm to RS256, which closes the algorithm-confusion trick where a verifier is talked into accepting a token that was signed some weaker way or not signed at all. That verification is the answer to the obvious question, which is what stops someone from skipping the login page and calling the Worker directly with a forged header. A forged token does not survive the check. Handing identity to a layer that does it for a living is one of the better decisions in the project, because the auth bugs I am most likely to write are the ones I now do not have to write.
The browser side runs under a tight content policy. The line that earns its keep is script-src 'self' with no unsafe-inline. There is no inline script anywhere in the app, so the policy can refuse all of it:
Content-Security-Policy:
default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline';
img-src 'self' https:; font-src 'self'; connect-src 'self' https://*.ingest.us.sentry.io;
object-src 'none'; base-uri 'self'; form-action 'self'; frame-ancestors 'none'An injected <script> tag or an onerror= attribute does not execute, because the page only runs code served from its own origin. frame-ancestors 'none' blocks clickjacking, and object-src 'none' with base-uri 'self' closes the older tricks.
The policy does relax in two places, and it is worth being straight about them rather than calling it airtight. img-src allows any https: host, because letting members embed an image from anywhere is a feature, not an oversight. And style-src allows 'unsafe-inline', because the frontend sets inline style attributes and there is no clean way around that. Inline styles are a much smaller risk than inline script. A style attribute cannot run code. The directive that can, script-src, is the one with no exceptions, and it is the one that has to be airtight. A strict policy does not mean no loosening anywhere. It means none on the part that can execute code, and a reason you can say out loud for the parts that cannot.
Anywhere user text reaches the database it goes through a bound parameter, never a glued-together string. The one risky surface is search, because full-text search has its own query language a person could try to abuse. So a search string never reaches the engine as typed. It is cut into bare alphanumeric words, each quoted as a literal, and recombined:
// letters, numbers, underscore only. max 12 terms, each quoted as a literal
const terms = (q.match(/[\p{L}\p{N}_]+/gu) ?? []).slice(0, 12);
const match = terms.map((t) => `"${t}"`).join(" ");An attacker’s quotes, operators, and column filters do not survive that, because anything that is not a letter or a number is not a term. The only values I ever paste straight into SQL are integer ids that came from my own database, parsed by a function that rejects anything non-numeric, and an integer cannot carry an injection.
Uploads are their own surface, and the most dangerous one, because a file the browser renders inline can carry script. So an upload is never trusted to be what it claims. The server ignores the browser’s stated content type and reads the file’s own magic bytes. A JPEG, PNG, GIF, or WebP keeps its real type and serves inline, and a PDF serves inline. Anything else, an SVG or an HTML file wearing an image’s name, is reclassified to a generic binary and served as a forced download, so it can never run as script on the forum’s own origin. That one rule closes the classic “upload an SVG, get stored XSS” hole.
Two more things happen on the way in. Images have their EXIF stripped, so a member’s photo does not quietly publish the GPS coordinates of where it was taken. And the filename is cut down to something safe, with control characters and line breaks removed, because the filename rides into a response header and a stray newline there is a header-injection trick.
The files are never public. R2 sits behind a proxy in the Worker, and a request for an image checks that the viewer is actually allowed to see it: staff, or the uploader, or an avatar, or a post that embeds the image and lives in a category the viewer can read. Fail that and the answer is a 404, not a 403, so the proxy does not even confirm the file exists. An image in a private thread is as private as the thread.
There are smaller guards underneath, and the reasoning behind two of them is the interesting part.
Every mutating request has to carry a custom header that only the app’s own code sets:
That one line closes the classic cross-site request trick. A malicious page can make your browser fire a POST at my API, but it cannot add a custom header to that request, so the request is rejected before it reaches any handler. A same-origin check and a JSON-content-type requirement sit alongside it as backstops.
Every member also has a per-day write ceiling, and the check is a single statement so two concurrent requests cannot both slip under it:
INSERT INTO write_counters (user_id, day, action, count) VALUES (?1, ?2, ?3, 1)
ON CONFLICT(user_id, day, action) DO UPDATE SET count = count + 1 WHERE count < ?4If the conflicting update changes no row, the ceiling was already reached, and the write is refused with a 429. The increment and the check are the same atomic operation, so there is no window between them for a flood to exploit. The slot is reserved before the write and refunded if the write fails, which keeps the counter honest without an interactive transaction, something D1 does not offer.
The unsubscribe links in emails are signed with an HMAC, so they work without a login but cannot be forged. The link itself only validates the token and shows a confirm button. It does not unsubscribe on its own, because mail clients and security scanners follow links in the background, and a GET that mutated would unsubscribe people who never clicked anything. The real unsubscribe is the POST behind the confirm button. The bounce webhook checks its signature before it believes a word of the payload.
That is the posture. Identity at the edge, a content policy strict enough to refuse inline script, bound parameters with the one risky surface reduced to bare words, a custom header on every write, ceilings that cannot be raced, and signed links that refuse to mutate on a background GET. Then there is the one piece of standard advice I looked at and decided against.
HSTS: The One I Left Out
While I was hardening things, I checked the security headers. They were already strong. A tight content policy, no inline scripts, clickjacking blocked, the usual list. One thing was missing, and the missing thing turned out to be a better example than anything that was present.
There was no HSTS header. HSTS tells a browser to refuse plain HTTP for your domain, which closes a real gap. The standard advice is to add it. For most sites, add it.
I have about fifteen subdomains under this domain.
HSTS is one of the few web headers that is hard to take back. The browser remembers it for as long as you told it to, and the option that extends it to every subdomain will break any subdomain that is not fully on HTTPS, with no quick undo. The Cloudflare dashboard switch for it is domain-wide and has exactly that option. For someone with fifteen subdomains, that switch is a foot-gun.
A version scoped to only the forum’s own hostname would be safe, because it cannot touch the other subdomains. But the real-world gain over the redirect-to-HTTPS that is already in place, for a forum that already sits behind a login wall, is small.
So I left it off. On purpose. “Obviously add HSTS” is the right advice for most sites and the wrong advice for this one.
None of This Was Clever
That is the whole of the engineering. The performance work was a cache, a stored counter, a handful of parallel queries, and one extra join. The scale work was a flag and a fallback. The security work was mostly confirming things were already fine and then deciding, with a reason, to skip the one thing that was missing.
The hard part was never the code. On a platform that bills by the row, the query that looks free on the latency graph can be the most expensive thing you run. You do not fix that by making it faster. You fix it by not running it. But knowing that only mattered because the query was mine to change.
I could have skipped all of it. I could have rented a small server, installed a forum that already exists, and had something running in a weekend. It would have worked. But I would have owned a config file and a monthly bill and not much else. I could not have changed how a count is stored, or folded an unread flag into a query, or decided that a slightly stale cache was a fair trade at a quiet hour, because none of those knobs would have been mine to turn.
That is the trade, and I think it is the right one. Owning the whole stack means living inside its limits. A database that charges by the row. A runtime with no memory between requests. A free tier whose edges are hard and visible. Those limits are not the price of ownership. They are the thing that made the work worth doing, because every one of them forced a decision I had to understand. A rented stack would have made those calls for me, and I would never have known they were calls at all.
The training groups I run now have somewhere to post tidbits, useful links, and longer arguments without cluttering up a group chat, and what they write there stays theirs.
Thanks for reading,
Jesse
Comments