Empirical Systems Performance
Inference Performance Benchmarks
Measured benchmarks comparing EverestQ Engine Native Core against llama.cpp, vLLM, and SGLang.
| INFERENCE ENGINE | TARGET MODEL | THROUGHPUT | TTFT (MS) | PEAK VRAM |
|---|---|---|---|---|
| EverestQ Engine (Native) | everestq-llama3 (8B Q4_K_M) | 84.50 tok/s | 12.40 ms | 5,600 MB |
| llama.cpp (b3200) | llama3-8b-q4_k_m | 79.20 tok/s | 14.10 ms | 5,800 MB |
| vLLM (v0.6.0) | llama3-8b-fp16 | 71.00 tok/s | 18.50 ms | 16,000 MB |
| SGLang (v0.3.0) | llama3-8b-fp16 | 73.40 tok/s | 17.80 ms | 15,800 MB |