Latency vs Throughput

← Back to Fundamental Tradeoffs

Batching improves throughput but increases per-request latency. Optimizing for one often comes at the expense of the other. Real-time systems prioritize latency; batch processing systems prioritize throughput.

Key Properties

• Batching Tradeoff
• Queue Depth
• Response Time Percentiles

Related

• Scalability Review (capacity planning)

---

tradeoffs latency throughput