How to Scale a Web App Without Overengineering It

Imagine a wholesale buyer opening a product page. The buyer can tolerate a description that is a few minutes old. They cannot tolerate the site selling stock that is already gone. The order may appear immediately even if a warehouse still needs time to assign it.

These actions look like one page, but they carry different promises. A web app scales when it can keep those promises as more people, data, and failures put it under pressure. The goal is not to add impressive infrastructure. It is to find which promise is breaking and make the smallest change that protects it.

To scale a web app without overengineering it:

1. Pick the user action that is failing.
2. Define “working” with a number: speed, errors, work completed, data age, or completion time.
3. Find where the request spends its time or meets a full resource.
4. Make the smallest reversible change that relieves that bottleneck.
5. Write down the trade-off the change creates—for example, older data, delayed work, duplicate attempts, or a new failure point.
6. Decide how failure will be detected, who responds, and how the system recovers.
7. Test the same user action again.

Sometimes the answer is ordinary: repair a slow query, simplify a workflow, add an index, or use a larger machine. Measured pressure—or an explicit reliability, isolation, or data-placement requirement—should justify a cache, queue, extra application copies, or database sharding. Sometimes the correct decision is to change nothing.

Before choosing infrastructure for the wholesale portal, separate what the page owes the buyer.

For inventory, one record must have the final say about whether units are available. Engineers call this the authoritative state. The reservation step must check and update that record in a way that prevents two buyers from claiming the same unit. Databases offer transactions, locks, and conditional updates, but the exact behavior must be tested rather than inferred from the word “transaction.” PostgreSQL’s transaction-isolation documentation shows why that choice matters.

Correctness is narrower than “never show cached inventory.” The page may show a recent estimate for browsing, then validate the final quantity when the buyer reserves it. The display path and decision path can have different guarantees.

Descriptions, images, and historical reports may tolerate bounded staleness. The acceptable age should be a product decision: perhaps minutes for copy, seconds for a dashboard, and an authoritative check for the reservation decision.

Without that bound, “use a cache” and “read from a replica” leave the visible behavior undefined.

The buyer needs to know whether an order was accepted. A warehouse allocation, partner webhook, or PDF document may be allowed to finish later. The interface must distinguish an accepted operation from a completed one and provide a way to reach a final state after failure.

Every system has a point at which more work arrives than it can complete within its target. The portal needs an overload decision: wait, reject, rate-limit, shed lower-priority work, or accept a bounded backlog. Continuing to accept everything can turn a short peak into a long recovery.

Data that must be checked together is usually cheaper and safer to keep together. One customer account’s inventory, permissions, and reservations may belong in the same place, while reports across all customers may then become harder. A placement rule that helps one workflow can make another more expensive, so data locality must follow real query and isolation needs.

These five promises form the product brief. The next job is to turn a vague complaint into measurable pressure. Infrastructure enters the discussion only after that.

“The portal is slow” does not identify a decision. Which action is slow? Under what demand? How slow is outside the target? Is the system busy, waiting, blocked, or calling something else?

Google’s site reliability guidance calls a measured behavior—such as speed or errors—a service-level indicator. The boundary a team aims to stay within is its service-level objective. The useful measurement depends on the promise: a checkout may care about correctness and duplicate prevention, while a search page may accept a slightly older result to respond faster.

A practical starting sentence is:

During [one user action], [one measurement] misses [the agreed target] under [an observed condition], while [a measured bottleneck] is full or waiting.

Consider this explicitly hypothetical, synthetic scenario for the wholesale portal. “Exports are slow” is vague. A better pressure statement is:

The target is for 95 of every 100 large exports to finish within four seconds. When 150 customers request one at once, that cutoff rises to 38 seconds. Export workers are full; ordinary pages still meet their targets.

Engineers call that cutoff the 95th percentile, or p95. It shows more of the slow end than an average, although the slowest 5%, failures, and different request types still need separate inspection.

The improved statement points toward export work. It does not justify scaling the whole application.

The same evidence can support different answers depending on the promise. If buyers need only a quick acknowledgement, a queue and status page may help. If the file itself must finish within four seconds, a queue merely hides the wait; the team must reduce the work or add worker capacity.

Think of an export as a parcel moving through stations: browser, application, export worker, database, and file storage. The buyer sees the total delay; the team needs the delay at each station. If export workers are full while the database is mostly idle, more database capacity will not help.

Useful monitoring connects what users experience with the resource or dependency causing it. Google SRE’s monitoring guidance highlights latency, traffic, errors, and saturation as core signals. Saturation means a resource is operating near its useful limit; contention means several tasks are waiting for the same resource. A real diagnosis usually needs domain signals too: database lock time, connection wait time, queue age, replica lag, cache hit rate, or failures from a partner API.

Follow the affected journey across its handoffs:

1. Observe the user journey. Record end-to-end latency, error rate, and the volume and shape of demand. A steady 100 requests per second is different from 6,000 arriving in ten seconds.
2. Break down the wait. Measure application work, database time, connection waiting, cache or network calls, and outside dependencies. A fast application cannot compensate for a dependency that consumes most of the deadline.
3. Check saturation and contention. CPU, memory, worker slots, database connections, locks, I/O, and provider quotas can all impose different limits.
4. Reproduce the hypothesis safely. Use a representative load and data shape in an appropriate environment. Change one relevant variable and see whether the user-facing target improves.
5. Keep the counter-evidence. If doubling application capacity leaves latency unchanged while database wait time rises, the test has ruled out one attractive answer.

Registered-user totals are not capacity requirements. A large inactive account base may create almost no work, while a few simultaneous analytical queries may consume a database. Capacity depends on what arrives together and what each request does.

The evidence may point to code, a query, connection waiting, application capacity, an outside dependency, or the design of the user journey itself. That distinction matters more than an architecture vocabulary test.

A scaling change is incomplete if its record contains only the benefit. Before implementation, write eight lines:

• Promise: Which user-facing result does this protect?
• Evidence: Which measurement identifies the present constraint?
• Intervention: What is the smallest change being proposed?
• Changed guarantee: What may now be stale, pending, duplicated, reordered, rejected, or temporarily unavailable?
• Detection: Which metric and alert expose failure?
• Authority and owner: Who may accept the trade-off, and who responds when it fails?
• Recovery: How does the system degrade, reconcile, and return to a healthy state?
• Exit: What triggers rollback, scale-down, replacement, or removal?

For example, moving warehouse allocation to a queue may protect checkout speed, but it changes “done” to “accepted.” The team now owes buyers a pending status, must prevent duplicate shipments, monitor old jobs, name who reconciles failures, and define when to bypass or remove the queue.

This record turns an infrastructure idea into an accountable decision. It also gives engineers and coding tools clear boundaries: what may change, how success is tested, and when to stop. Tools can profile, test, and implement; named people still decide whether stale inventory, duplicate work, or longer recovery is acceptable.

For a reusable way to record goals, boundaries, proof, stop conditions, and rollback before an AI coding agent changes the system, use How to Write a Clear Specification for an AI Coding Agent.

The following mechanisms are options inside the five-promise brief. They are not stages that every application completes.

Start with the work required to make the correct reservation. A slow authoritative path can have several causes.

A database connection is not free. PostgreSQL documents a configured connection limit and the resource implications of raising it. A connection pool reuses a bounded set of server connections and can make excess clients wait rather than all opening a new one. PgBouncer’s pool modes and limits show why the details matter: connection reuse changes according to whether pooling is scoped to a session, transaction, or statement.

Each running copy of an application can have its own connection pool. Ten copies with 20 connections apiece may still ask the database for about 200. A database proxy can bound or share server connections across many clients, but neither kind of pool creates database CPU, I/O, or lock capacity.

Next inspect the query. PostgreSQL’s index documentation explains the basic trade: an index can make selected reads faster, while consuming space and adding maintenance to writes. The planner may decline to use it. Verify the access path and the measured workload rather than assuming that the existence of an index settles the question.

If a single database still cannot meet measured throughput, maintenance, isolation, or locality needs after simpler work, sharding may become a candidate. Sharding splits records across independently managed partitions, often on different database nodes. The shard key is the rule that decides where each record lives. That rule must spread both data and traffic, support common queries, and remain changeable without a dangerous migration. MongoDB’s shard-key guidance documents the distribution and query questions a team must test.

Treat sharding as a response to a demonstrated database limit, not a default milestone. Query repair, archiving, table partitioning, a larger database, or separating workloads often solve the same pressure with fewer operating duties. Verified isolation, geography, or data-residency requirements may also justify sharding. If sharding is necessary, use the database’s supported resharding process or a carefully designed routing approach; changing a simple remainder-based rule can otherwise move most records at once.

Once the authoritative reservation path is protected, decide which other reads may lag.

In the common primary-and-replica topology, a read replica is a database copy used for eligible reads. Amazon RDS documents asynchronous replica updates, which means the replica can trail the writer. It can increase read capacity but does not generally remove pressure from primary writes.

A historical report may tolerate lag. A page shown immediately after an address change may need the writer or a database-specific read-after-write mechanism. Monitor replica lag against the product’s freshness bound; a path that normally meets the bound can fall outside it during an incident.

A cache saves a result closer to the next request or avoids repeated computation. In cache-aside, the application checks the cache, loads the source on a miss, and stores the result for later calls. Microsoft’s cache-aside guidance emphasizes expiration, eviction, and the risk of stale values after the source changes.

Ask two questions for each cached value:

1. May it be displayed when it is old?
2. May it authorize an irreversible action when it is old?

For the catalogue, record the maximum acceptable age, invalidation or version rule, miss behavior, outage behavior, and evidence that repeated reads save enough work to justify the cache. Keep the reservation decision on the authoritative path unless a stronger design proves the required correctness.

A queue is a waiting line between the site and a worker. When its durability is configured and verified, it can let the site say “we received it” before the worker finishes, absorb a burst, or separate two parts of a system with different capacity. It also changes the meaning of success.

If the portal responds after accepting a warehouse-allocation request, the allocation is not complete. It is pending. HTTP defines 202 Accepted for a request accepted for processing when that processing has not finished. The interface and API need a status model, a final success or failure state, and a recovery path.

Delivery behavior depends on the system and configuration. Amazon SQS Standard queues document at-least-once delivery and best-effort ordering, so a message can be repeated or arrive out of order. Processing the same message twice must not create two shipments; engineers call this idempotency. It requires durable, end-to-end controls around the real side effect, not merely a worker described as “idempotent.”

Retries need limits and spacing. Persistently failing work often moves to a dead-letter queue, which also needs monitoring, diagnosis, and a policy for retrying messages safely after correction. It is a quarantine area, not a repair mechanism.

Protect the transaction boundary too. If the order commits and publishing the allocation event fails, the order exists but downstream work can disappear. If publication succeeds before a rollback, a worker may act on absent business state. The transactional outbox pattern stores an event with the business transaction, then relays it. Relays can still duplicate delivery, so consumers need the idempotency controls.

Measure the age of the oldest relevant work, end-to-end completion time, retries, dead letters, worker saturation, and accepted operations that never reach a terminal state. A raw queue count can hide a small number of very old and consequential jobs.

Only after identifying the saturated resource does “more capacity” become a testable proposal.

Vertical scaling makes one machine bigger. Horizontal scaling adds more machines. A load balancer decides which healthy machine receives each request. NGINX and AWS Elastic Load Balancing document multiple routing algorithms and health behaviors; a load balancer does not universally divide work evenly.

A larger machine can be a quick, reversible experiment. It may retain a single failure boundary or hit cost and instance limits. More application instances can increase aggregate capacity. Improving availability additionally requires suitable failure separation, health routing, and tested failover. Neither approach multiplies a shared database, provider quota, network, or budget.

For requests to move among machines, stateless handling means no running app copy is the only home of information the product promised to keep. Temporary buffers, local connection pools, and disposable caches can still exist; the app copy must simply be able to disappear without losing durable user state.

A login session still needs its own security and availability decision. It may live in a signed browser token, a shared server store, or a database, or traffic may be kept on one app copy. The right choice depends on how quickly access must be withdrawn and what happens when that copy or store fails.

Finally, define overload behavior. Adding instances without admission control can transfer an unbounded peak to the database or another dependency. Google SRE’s overload guidance explains why a system needs to reject excess work deliberately instead of accepting traffic it cannot finish usefully. A scaling plan should say which work waits, which work is rejected, how retry storms are limited, and which higher-priority journeys retain capacity.

Return to the wholesale portal. Each proposal now has a specific approval test:

• Cache: what may be stale, for how long, how it refreshes, and what happens if it fails.
• Queue: what “accepted” means, how duplicates are prevented, how old work is found, and who reconciles failure.
• More app capacity: proof that the app itself is full, protection for shared dependencies, and a deliberate overload rule.
• Sharding: proof that the key spreads real load, supports important queries, and solves a limit that simpler changes cannot.

Approve a scaling change when its record connects a user promise to measured evidence, identifies the altered guarantee and new failure mode, assigns an owner, and provides recovery and exit conditions.

Until those facts exist, the proposal is an infrastructure idea. Once they do, the team has a decision it can test, operate, challenge, and reverse.

• Google, Site Reliability Engineering: Service Level Objectives, accessed 2026-09-06.
• Google, Site Reliability Engineering: Monitoring Distributed Systems, accessed 2026-09-06.
• Google, Site Reliability Engineering: Handling Overload, accessed 2026-09-06.
• NGINX, Using nginx as HTTP load balancer, accessed 2026-09-06.
• Amazon Web Services, How Elastic Load Balancing works, accessed 2026-09-06.
• PostgreSQL Global Development Group, Connections and Authentication, accessed 2026-09-06.
• PgBouncer, Features, accessed 2026-09-06.
• PostgreSQL Global Development Group, Indexes, accessed 2026-09-06.
• PostgreSQL Global Development Group, Transaction Isolation, accessed 2026-09-06.
• Amazon Web Services, Working with DB instance read replicas, accessed 2026-09-06.
• Microsoft Azure Architecture Center, Cache-Aside pattern, accessed 2026-09-06.
• Amazon Web Services, Amazon SQS queue types, accessed 2026-09-06.
• Amazon Web Services, Using dead-letter queues in Amazon SQS, accessed 2026-09-06.
• Amazon Web Services, Transactional outbox pattern, accessed 2026-09-06.
• IETF, RFC 9110 — 202 Accepted, published 2022-06; accessed 2026-09-06.
• MongoDB, Choose a shard key, accessed 2026-09-06.
• MongoDB, Reshard a collection, accessed 2026-09-06.
• Amazon Web Services, Consistent hashing, accessed 2026-09-06.