Load Testing

Types of Load Tests

Baseline: measure normal behavior. What are the P50/P99 latencies at expected traffic?

Stress: increase load until something breaks. Find the system’s limits.

Soak: run at expected load for hours. Find memory leaks, connection pool exhaustion, log file growth.

Spike: sudden 10x traffic jump. Does the system recover? How long does it take?

Breakpoint: same as stress but you’re looking for the exact RPS where P99 crosses your SLO.

autocannon

Fast, simple HTTP benchmarking for Node.js:

# 100 connections, 30 seconds
npx autocannon -c 100 -d 30 http://localhost:3000/api/orders

# With a body (POST)
npx autocannon -c 100 -d 30 \
  -m POST \
  -H 'Content-Type: application/json' \
  -b '{"customerId":"cust-123","items":[{"productId":"prod-456","quantity":1}]}' \
  http://localhost:3000/api/orders

# Ramp up (10 connections, then 50, then 100)
npx autocannon -c 10 -d 10 http://localhost:3000/api/orders
npx autocannon -c 50 -d 10 http://localhost:3000/api/orders
npx autocannon -c 100 -d 10 http://localhost:3000/api/orders

Output:

Stat         | 2.5% | 50%  | 97.5% | 99%  | Avg   | Stdev | Max
Latency      | 14ms | 22ms | 89ms  | 145ms| 23.4ms| 19.1ms| 2341ms

Req/Sec      | 3240 | 4100 | 4380  | 4410 |
Bytes/Sec    | 2.1M | 2.7M | 2.9M  | 2.9M |

35672 requests in 10s, 236 MB read

Watch for: P99 climbing, error rate appearing, Max exploding above P99 (outliers).

k6

k6 is a scripting tool for complex load test scenarios — think realistic user journeys, not just “hammer this endpoint”:

// load-test.js
import http from 'k6/http';
import { check, sleep } from 'k6';
import { Rate } from 'k6/metrics';

const errorRate = new Rate('errors');

export const options = {
	stages: [
		{ duration: '2m', target: 10 }, // ramp up to 10 users
		{ duration: '5m', target: 10 }, // stay at 10
		{ duration: '2m', target: 50 }, // ramp to 50
		{ duration: '5m', target: 50 }, // stay at 50
		{ duration: '2m', target: 100 }, // ramp to 100
		{ duration: '5m', target: 100 }, // stay at 100
		{ duration: '2m', target: 0 } // ramp down
	],
	thresholds: {
		http_req_duration: ['p(99)<500'], // 99% of requests < 500ms
		errors: ['rate<0.01'] // error rate < 1%
	}
};

const BASE_URL = 'http://staging.api.example.com';

export default function () {
	// Realistic user journey
	// 1. Login
	const loginRes = http.post(
		`${BASE_URL}/auth/login`,
		JSON.stringify({
			email: 'test@example.com',
			password: 'testpass'
		}),
		{ headers: { 'Content-Type': 'application/json' } }
	);

	check(loginRes, {
		'login succeeded': (r) => r.status === 200
	});

	const token = loginRes.json('token');

	// 2. Browse products
	const productsRes = http.get(`${BASE_URL}/api/products`, {
		headers: { Authorization: `Bearer ${token}` }
	});

	check(productsRes, {
		'products loaded': (r) => r.status === 200,
		'has products': (r) => r.json('items').length > 0
	});

	errorRate.add(productsRes.status >= 400);
	sleep(1); // user "thinks"

	// 3. Create order
	const product = productsRes.json('items')[0];
	const orderRes = http.post(
		`${BASE_URL}/api/orders`,
		JSON.stringify({
			items: [{ productId: product.id, quantity: 1 }]
		}),
		{
			headers: {
				Authorization: `Bearer ${token}`,
				'Content-Type': 'application/json'
			}
		}
	);

	check(orderRes, {
		'order created': (r) => r.status === 201
	});

	errorRate.add(orderRes.status >= 400);
	sleep(2);
}

# Run the test
k6 run load-test.js

# With output to InfluxDB (for Grafana dashboard)
k6 run --out influxdb=http://localhost:8086/k6 load-test.js

# HTML report
k6 run --out json=results.json load-test.js
k6-html-reporter results.json

Finding the Breaking Point

Binary search on RPS to find where P99 crosses your SLO:

// breakpoint-test.js
import http from 'k6/http';
import { check } from 'k6';

export const options = {
	executor: 'ramping-arrival-rate', // RPS-based (not VU-based)
	stages: [
		{ target: 100, duration: '1m' }, // 100 RPS for 1 minute
		{ target: 200, duration: '1m' }, // 200 RPS
		{ target: 400, duration: '1m' }, // 400 RPS
		{ target: 800, duration: '1m' }, // 800 RPS
		{ target: 1600, duration: '1m' } // 1600 RPS — will this break it?
	],
	preAllocatedVUs: 200,
	maxVUs: 500,
	thresholds: {
		http_req_duration: ['p(99)<500']
	}
};

Watch the Grafana dashboard while the test runs. The point where P99 starts climbing steeply is your inflection point — where queuing begins. Your sustainable RPS is about 70% of the breaking point.

Realistic Test Data

Load testing with cust-123 hardcoded produces unrealistic cache hit rates and database behavior:

// k6 — parameterized test data
import { SharedArray } from 'k6/data';

const users = new SharedArray('users', function () {
	return JSON.parse(open('./test-users.json')); // 10,000 test users
});

const products = new SharedArray('products', function () {
	return JSON.parse(open('./test-products.json'));
});

export default function () {
	const user = users[Math.floor(Math.random() * users.length)];
	const product = products[Math.floor(Math.random() * products.length)];

	// Now the test exercises different code paths, database queries,
	// and cache keys — closer to production behavior
}

Monitoring During Load Tests

Watch these metrics while the test runs:

# Node.js process
watch -n1 "node -e \"const p=process; console.log(JSON.stringify(p.memoryUsage()))\""

# PostgreSQL connections and queries
watch -n1 "psql -c \"SELECT count(*), state FROM pg_stat_activity GROUP BY state\""

# Redis
watch -n1 "redis-cli info stats | grep -E 'connected_clients|used_memory_human|instantaneous_ops_per_sec'"

# System
htop  # CPU and memory
iostat -x 1  # disk I/O
ss -s  # connection counts

When latency spikes: check which resource saturated first. CPU? Disk I/O? Connection pool? The first one to saturate is the bottleneck.

Staging vs Production Differences

Load test in staging, but be aware of the gaps:

	Staging	Production
Database size	Small (100k rows)	Large (10M+ rows)
Cache state	Cold	Warm
Index effectiveness	Artificially good	Real-world performance
External API latency	Mocked	Variable
Background jobs	Off	Running and consuming resources

Mitigation:

• Seed staging with production-scale data (anonymized)
• Run load test with cache cold AND warm — measure both
• Enable background jobs during load test
• Mock external APIs with realistic latency (p50=100ms, p99=500ms)

A load test on empty-table staging will not reveal the index you forgot. Test with real data scale.