Scaling Rails: Vertical & Horizontal

When to use it: deciding how to scale a Rails app under load — choosing
between a bigger box and more boxes, sizing Puma workers/threads, relieving the
database, or planning read replicas / sharding. Use it to scale to the next
measured bottleneck, not preemptively.

The one-line model: scaling is a sequence, not a single lever. Make the box
bigger until you hit a physical ceiling, then add stateless boxes behind a load
balancer — but the real bottleneck is almost always the database, so most of the
work is there. Diagnose, fix the current bottleneck, measure again.

1. Vertical scaling — do this first

• What: add RAM / faster CPU / disk to one server. Simplest possible step, no
  architecture change.
• Why first: cheapest in engineering time; buys runway while you find the real
  constraint.
• The ceiling is real. You cannot buy ever-bigger hardware forever. Shopify hit
  exactly this: "In 2015, it was no longer possible to continue buying a larger
  database server," which forced them into sharding.¹
• Ruby-specific vertical wins (verify against your versions — under-documented in
  this research): enable YJIT (default-on territory in modern Ruby 3.3+/Rails 8),
  and tune the allocator (jemalloc or MALLOC_ARENA_MAX) to cut memory bloat.
  Treat the specific numbers as something to benchmark yourself.

2. Horizontal scaling — more boxes

• What: run the app on several machines at once. Adds throughput and
  fault tolerance — if one server dies, the others keep serving.²
• Fault tolerance is not automatic. Multiple machines only survive failures if a
  load balancer with health checks routes around the dead node, and the app
  servers are stateless so any node can serve any request (verifier caveat).
• Statelessness: keep no per-server state — sessions in signed cookies or a shared
  store, not local memory; uploads to object storage, not local disk. (Exact
  sticky-vs-stateless session mechanics were under-evidenced in this research — verify
  your session strategy before relying on it.)

3. Puma: processes vs threads, and the GVL

The defining constraint of MRI/CRuby:

• The GVL lets only one thread run Ruby at a time, so a single Ruby process never
  uses more than one CPU core, no matter how many threads it has.³
  → For CPU parallelism you need multiple processes (Puma workers).
• Threads help only during I/O waits (DB queries, HTTP calls, Redis release the
  GVL). A well-crafted Rails app spends under 50% of its time on I/O, so it's unlikely
  to benefit from more than ~3 threads — which is why the Rails default dropped from 5
  to 3 in 7.2.³ I/O-heavy apps (lots of external API calls) can
  justify 5–10.
• Sizing heuristic: workers ≈ CPU cores, threads = 3 (default), then measure.
  An extra process costs real memory (only partly shared via copy-on-write); an
  extra thread is cheap but only pays off on I/O.³

4. The database is the real bottleneck

"It's easy to scale [workers] to the point that you're overloading your
database."⁴ Redis rarely is the problem; the DB usually is.

• Connection pooling. PostgreSQL forks a separate OS process per connection
  (~5–10 MB each) — far more expensive than a thread.⁵ So put a pooler
  (PgBouncer) in front of Postgres. Adopt pooling based on measured connection
  pressure — there is no fixed "150 clients" threshold (that claim was refuted).
• Pool sizing: Rails defaults the AR pool to RAILS_MAX_THREADS, and Sidekiq uses
  the same env var, so the pool stays matched to concurrency if you let it.⁵
• Read replicas (Rails multi-DB): offload reads with connects_to:

  connects_to database: { writing: :primary, reading: :primary_replica }
  

  Standard since Rails 6, unchanged through 8.⁶
  Caveat (refuted claim): Rails does not auto-route writes to the primary by
  HTTP verb out of the box — automatic role switching needs explicit configuration.
• Horizontal sharding — last resort. connects_to shards exists, but swapping is
  manual application work via connected_to(role:, shard:).⁶
  Shard only when a single DB can no longer be made bigger (Shopify's 2015
  trigger).¹ Tooling at scale: Vitess routes by a sharding key —
  queries with the key hit one shard; queries without it become expensive
  scatter queries across all shards, so pick a key most queries include.⁷
  Shopify's "pods" go further: isolated groups of shops each with a fully isolated
  datastore set, so one pod's outage stays contained.¹

5. Background jobs — offload the slow work

• Move anything slow/retryable/fan-out off the request cycle.
• Sidekiq: threads in one process (default 5 in v7+, was 10 in v6). Run one
  Sidekiq process per container; don't assume you need multi-process (Swarm is a paid
  add-on).⁴ Best for very high Redis-backed throughput.
• Solid Queue: DB-backed Active Job backend (Postgres/MySQL/SQLite via
  FOR UPDATE SKIP LOCKED), no Redis, the Rails 8 default — replaces
  Resque/Delayed Job/Sidekiq "for most people."⁸⁹ Proven scale:
  ~20 million jobs/day at HEY.⁹
• Keep jobs idempotent (retries happen) and small.

6. Caching layers

• Solid Cache (Rails 8 default): disk-backed Active Support cache replacing
  Redis/Memcached for fragment caches. Trade-off measured at 37signals: reads ~40%
  slower, but the cache is ~6× larger on storage ~80% cheaper; Basecamp p95 fell
  375 ms → 225 ms.¹⁰⁹ SSD latency is fine for fragment caching.
• Russian-doll fragment caching for nested view fragments; HTTP/CDN caching at
  the edge; Redis still wins when you need genuine sub-millisecond in-memory reads.

7. Rails 8 / 2025–2026: the "no PaaS" stack

• The Solid trio — Queue, Cache, Cable — removes the Redis dependency for jobs,
  cache, and websockets, all DB-backed.⁹
• Kamal 2 ships preconfigured: deploy to a cloud VM or your own hardware,
  "production in under two minutes" (best-case, servers already provisioned).⁹
• Net effect: a credible own-hardware stack without managed Redis/PaaS — the
  trend this app already follows (Kamal + SQLite).

8. Diagnose, don't pre-optimize

Scale to the next measured bottleneck:

• N+1 queries: detect with Bullet / strict_loading; fix with includes.
• APM: New Relic / Scout / Datadog to find the actual slow path (DB? view? external
  call?) before adding capacity.
• Load test to learn whether you're CPU-bound (→ more workers), I/O-bound (→ more
  threads), or DB-bound (→ pooling/replicas/caching).

(This research surfaced little concrete tooling detail here — treat the specific tools
as a starting list, not a verified recommendation.)

Decision cheat-sheet

Symptom	First move
One box maxed, simple app	Vertical: bigger box, YJIT, malloc tuning
CPU-bound, cores idle	More Puma workers (≈ cores)
Slow on external I/O	A few more threads per worker
Need throughput + resilience	Horizontal: stateless app servers + LB + health checks
DB connections saturated	PgBouncer + pool sized to RAILS_MAX_THREADS
Read-heavy	Read replica via connects_to
Single DB can't grow	Shard (connects_to shards / Vitess) — last resort
Slow requests doing heavy work	Background jobs (Solid Queue / Sidekiq)
Repeated expensive renders/queries	Caching (Solid Cache / fragments / CDN)

Caveats & things to verify yourself

• Version-sensitive defaults: Puma threads 5→3 (Rails 7.2); Sidekiq concurrency
  10→5 (v6→v7). Check your exact versions.
• Two refuted claims — don't rely on them: (1) Rails auto-routing writes to the
  primary by HTTP verb (needs explicit config); (2) a fixed "~150 clients" PgBouncer
  threshold (size to measured demand).
• Self-reported metrics: HEY's 20M jobs/day and Basecamp's cache numbers are
  first-party figures from the tool's makers — directionally credible, not independent
  benchmarks.
• Under-evidenced in this research: YJIT/jemalloc tuning specifics, sticky-vs-
  stateless session mechanics, and APM/load-test tooling — confirm before depending on
  them.

See also

• Rails Production Playbook — the broader engineering reference (scalability,
  observability, transactions, idempotency).
• Sandi Metz' Rules — keeping the code small enough to scale the team too.

Sources (primary)

• Rails Guides — Tuning Performance for Deployment (GVL, Puma sizing)
• Rails Guides — Multiple Databases (replicas, sharding)
• Rails 8: No PaaS Required (Solid trio, Kamal)
• 37signals — Introducing Solid Queue · Solid Cache
• Sidekiq — Scaling Sidekiq
• Shopify Engineering — Pods architecture · Scaling Shop with Vitess
• Percona — PgBouncer connection pooling