lokalni.ailokalni.ai⁄Stejný vstup, stejný výstup
identical to the referencediffers from the reference
LFM2.5-350Mdense, BF16, A100Qwen3-4Bdense, BF16, A100Qwen3.6-35B-A3BMoE, NVFP4, Blackwell
hellofactstorymathcodehellofactstorymathcodehellofactstorymathcode
E1batch size 1, one processreference: repeat 0 of the same setting
vLLMgreedy
vLLMsampling, T = 0.7
vLLMsampling, T = 1.0
llama.cppgreedy
llama.cppsampling, T = 0.7
llama.cppsampling, T = 1.0
E2batchingreference: E1 greedy, repeat 0
vLLMbatch of 2
vLLMbatch of 4
vLLMbatch of 8
llama.cpp--parallel 2
llama.cpp--parallel 4
llama.cpp--parallel 8
E3vLLM batch-invariant modereference: batch-invariant batch 1, repeat 0
vLLMbatch of 1––
vLLMbatch of 2––
vLLMbatch of 4––
vLLMbatch of 8––
E4L40S instead of A100reference: E1 greedy on A100, repeat 0
vLLMgreedy on L40S–
llama.cppgreedy on L40S–
E3cost of batch-invariant modevLLM, Qwen3-4B, A100, tokens/s of the whole batch
defaultbatch-invariant
batch of 1
118
39
3.0× slower
batch of 2
156
49
3.2× slower
batch of 4
264
86
3.1× slower
batch of 8
512
173
3.0× slower

report|plan|repository