API Framework Benchmark for Data-Intensive Services

Detailed methodology and results

Supporting methodology, figures, and tables are rendered here as native page content with the same visual system as the rest of this website.

Generated 2026-03-16 18:04:13 Test environment: Docker (2 CPU / 512MB per container) Databases: PostgreSQL (Aurora), StarRocks (Iceberg), OpenSearch (AWS)

How to Read This Report

Key Terms

Recommendation: It depends on a team

With proper best-practice implementations, four frameworks achieved 0% error rates across all scenarios: Fiber (Go) , Fastify (Node.js) , BlackSheep (Python) , and Elysia (Bun) . The performance differences between them are driven by the databases, not the frameworks all produce nearly identical p50 latencies (235 282ms mixed). The right choice depends on a team's language expertise and ecosystem needs:

Already know Python? BlackSheep (0% errors, highest sustained throughput at 43 req/s, mature async ecosystem with asyncpg/aiomysql)

Already know TypeScript? Fastify (0% errors, fastest isolated query speed for PG/OS, huge npm ecosystem)

Want maximum raw speed + tiny containers? Fiber (Go) (0% errors, fastest health endpoint at 554 req/s isolated, 10MB Docker image)

Want fastest raw overhead? Elysia (Bun) (140K req/s overhead, 0% mixed/sustained errors, but 4% stress errors)

1. Scenario Overview

2. Visual Comparison

Throughput by Scenario (req/s)

Error Rate by Scenario (%)

Latency p50 p95 Mixed Scenario

Latency p50 p95 Stress Scenario

Isolated: Per-Endpoint Throughput (req/s)

Isolated: Per-Endpoint p50 Latency (ms)

3. Isolated Endpoint Tests

4. Mixed Workload Per-Endpoint Breakdown

5. Stress Test Per-Endpoint Breakdown

6. Key Findings Recommendations

• Four frameworks achieved 0% errors across all scenarios. Fiber (Go), Fastify (Node.js), BlackSheep (Python), and Elysia (Bun) all handled mixed, stress, and sustained workloads with zero failures. This is a materially different result from the initial tests; fixed implementation issues such as missing SSL, wrong query parameter parsing, and StarRocks LIMIT syntax were responsible for most of the earlier "framework" errors.
• The database is the bottleneck, not the framework. Under real workload, all 6 frameworks produce nearly identical p50 latencies (235 282ms mixed). The 100x difference between a 39ms OpenSearch lookup and a 14s PostgreSQL aggregation dwarfs any framework overhead. Optimizing database queries (materialized views, caching, indexing) will deliver far more impact than switching frameworks.
• BlackSheep (Python) is the surprise performer. With proper async drivers (asyncpg + aiomysql), BlackSheep delivered the highest sustained throughput (43.4 req/s) and 0% errors everywhere . Its earlier 69% error rate was entirely due to a query-parameter handling issue, not the framework itself.
• Fastify (Node.js) is the most consistent under load. 0% errors in all scenarios, fastest isolated PG lookups (281 req/s, 37ms p50), and the best p95 latency under mixed load (12.5s vs 28 60s for others). If a team knows TypeScript, this is the safest choice.
• Fiber (Go) has the lowest per-request overhead (225us p50 health, vs 1 2ms for others) and the fastest PG row lookups (291 req/s, 35ms p50). The 10MB Docker image is 20x smaller than Python/Node containers. But the performance gap vs Fastify/BlackSheep is small once databases are involved.
• Elysia (Bun) has the fastest raw throughput at 140K req/s overhead 5x faster than FastAPI. Under real database load it performs well (0% mixed/sustained errors), but shows 4% errors under extreme stress (500 VUs). Bun's single-threaded model struggles when many slow queries (PG agg wide) block simultaneously.
• Hono+Bun is the weakest contender. Despite 102K overhead throughput, it shows 11% mixed errors and 25% stress errors the worst among the six. Its single-threaded model combined with the pg/mysql2 drivers creates head-of-line blocking on slow queries. It also has a 56% error rate on PG Agg Wide isolated tests (the slowest query type).
• FastAPI (Python) is reliable but slow under pressure. 0% errors on isolated tests but 10 14% on load scenarios because asyncpg waits for slow queries up to the 60s timeout rather than failing fast. The p95 hitting exactly 60.00s confirms this. With lower timeout settings or query optimization, this would improve significantly.
• PostgreSQL aggregations need optimization. PG Agg Wide (6-month range) takes 9 14 seconds across ALL frameworks. This is a database issue: consider materialized views, pre-computed rollup tables, or moving heavy aggregations to StarRocks which handles them in 78 98ms.
• StarRocks paginated queries are universally slow (2 4s p50 across all frameworks). This is the StarRocks Iceberg connector overhead, not the framework. Consider caching frequent pagination results in Redis.
• Large responses cost 2 5x more. PG Paginated 100 rows (56 119ms) is 1.5 2x slower than 20 rows (37 77ms). OS Search 100 (1.3 2.4s) is 5 10x slower than 20 results (150 333ms). Implement strict pagination limits and consider response compression (gzip) for production.
• OpenSearch is consistently the fastest data source for lookups (39 60ms) and aggregations (42 62ms) across all frameworks. The framework choice has almost zero impact on OpenSearch performance.

7. Final Verdict