Performance benchmarks.
EQ Engine vs Ollama vs llama.cpp vs vLLM on identical hardware platforms.
Throughput Comparison (tok/s — Higher is better)
EQ Engine (Native Core)84.50 tok/s
llama.cpp (b3200)79.20 tok/s
Ollama (v0.3.6)74.10 tok/s
vLLM (v0.6.0)71.00 tok/s
SGLang (v0.3.0)73.40 tok/s
Detailed Latency & Memory Telemetry Table
| INFERENCE ENGINE | TARGET MODEL | TTFT (MS) | THROUGHPUT | PEAK VRAM | COLD START |
|---|---|---|---|---|---|
| EQ Engine (Native Core) | Llama-3 8B (Q4_K_M) | 12.40 ms | 84.50 tok/s | 5,600 MB | 42 ms |
| llama.cpp (b3200) | Llama-3 8B (Q4_K_M) | 14.10 ms | 79.20 tok/s | 5,800 MB | 120 ms |
| Ollama (v0.3.6) | Llama-3 8B (Q4_K_M) | 16.80 ms | 74.10 tok/s | 6,100 MB | 240 ms |
| vLLM (v0.6.0) | Llama-3 8B (FP16) | 18.50 ms | 71.00 tok/s | 16,000 MB | 1,200 ms |
| SGLang (v0.3.0) | Llama-3 8B (FP16) | 17.80 ms | 73.40 tok/s | 15,800 MB | 1,100 ms |