Building Scalable APIs With Node.js

Scalability is not about frameworks. It is about fundamentals.

Teams often ask which Node.js framework scales best. The honest answer: none of them save you from fundamental I/O and resource management. Express, Fastify, and NestJS all handle thousands of requests per second out of the box. What kills scalability is how your code uses databases, external services, and memory.

Real API scalability comes from boring fundamentals: connection pooling, caching, backpressure handling, and careful I/O management.

Connection pooling and database management

Every database connection has overhead. Opening and closing connections per request is the most common scalability killer we see.

What to do:

• Use a connection pool: Configure your ORM or database driver to maintain a pool of reusable connections. For PostgreSQL with pg, set pool size to 10-20 per instance.
• Set timeouts: Add connection, query, and idle timeouts. A slow query should not hold a connection forever.
• Monitor pool usage: Track active connections, waiting requests, and pool exhaustion. If requests are waiting for connections, you need more capacity or fewer queries.

Caching strategy

Not every request needs to hit the database. A proper caching layer can reduce database load by 60-80% for read-heavy APIs.

Three levels of caching:

• In-memory cache: For frequently accessed, rarely changing data (config, feature flags, user permissions). Use Node.js in-process caching with TTL.
• Redis cache: For shared state across instances. Cache query results, computed aggregations, and session data. Set appropriate TTLs and cache invalidation logic.
• HTTP cache headers: For public endpoints with stable responses, use Cache-Control headers and let CDNs and browsers handle caching.

Cache invalidation is the hard part. Use explicit invalidation (clear cache on write) and TTL-based expiration as a safety net.

Rate limiting and protection

An unprotected API will be abused, whether intentionally or accidentally. Rate limiting protects your service and ensures fair resource allocation.

Implement these protections:

• Per-client rate limiting: Track requests per API key or user. Use Redis for distributed rate limiting across instances.
• Global rate limiting: Protect the overall service. If total traffic exceeds capacity, return 503 instead of crashing.
• Idempotency keys: For write endpoints, require idempotency keys so accidental retries do not create duplicate records.
• Request validation: Validate and sanitize inputs early. Reject malformed requests before they reach business logic or database queries.

Horizontal scaling patterns

Vertical scaling (bigger server) has limits. Horizontal scaling (more servers) is the path to real growth.

Key requirements for horizontal scaling:

• Stateless instances: No local file storage or in-memory session state. Everything shared goes to Redis or the database.
• Load balancer: Distribute traffic across instances. Use health checks to route away from unhealthy instances.
• Shared state: Redis for sessions, cache, and pub/sub. PostgreSQL for persistent data. Both must be external to the Node.js instances.
• Graceful shutdown: When scaling down, drain active requests before terminating. SIGTERM handler with a 10-second drain period.

Observability: see where time goes

You cannot optimize what you cannot see. Every API request should be traceable from entry to response.

What to instrument:

• Request logging: Log method, path, status code, response time, and request ID for every request.
• External call tracking: Time every database query, Redis call, and third-party API request. Log slow calls separately.
• Error tracking: Capture errors with full stack traces and context. Use Sentry or similar for real-time alerting.
• Metrics: Track requests per second, p50/p95/p99 latency, error rate, and queue depth. Alert on anomalies.

The p99 latency number tells you more about your API health than any average.

Real results

We built an API for a logistics client that processes 50,000+ requests daily. The architecture: Node.js with Fastify, PostgreSQL with connection pooling, Redis for caching and rate limiting, behind a load balancer. Average response time: 45ms. P99: 180ms. Uptime: 99.95%.

Need help scaling your API?

We build and scale production APIs. If your Node.js API is struggling under load or you are preparing for growth, we can help.

Book a free strategy call with our engineering team.