Twelve small instruction-tuned models (0.27–4 B) on six practical skills, each tested in English and in Czech. Every model runs in BF16 with greedy decoding and thinking switched off.
all results Tool calling Structured output Summarization Classification Knowledge Instruction following hover a point for its value; every number is also in the table below functiongemma-270m 0.27B en 17.1 cs 9.6 50 100 Tool calling Structured output Summarization Classification Knowledge Instructions granite-4.0-h-350m 0.34B en 52.4 cs 29.3 50 100 Tool calling Structured output Summarization Classification Knowledge Instructions lfm2.5-350m 0.35B en 52.3 cs 20.3 50 100 Tool calling Structured output Summarization Classification Knowledge Instructions qwen3.5-0.8b 0.87B en 64.5 cs 47.4 50 100 Tool calling Structured output Summarization Classification Knowledge Instructions lfm2.5-1.2b 1.2B en 61.2 cs 26.2 50 100 Tool calling Structured output Summarization Classification Knowledge Instructions qwen3.5-2b 2.3B en 75.9 cs 66.1 50 100 Tool calling Structured output Summarization Classification Knowledge Instructions gemma-4-e2b 2B eff. (5.1B) en 79.1 cs 74.2 50 100 Tool calling Structured output Summarization Classification Knowledge Instructions smollm3-3b 3.1B en 73.9 cs 53.4 50 100 Tool calling Structured output Summarization Classification Knowledge Instructions tiny-aya-global 3.3B en 70.3 cs 67.9 50 100 Tool calling Structured output Summarization Classification Knowledge Instructions granite-4.2-3b 3.7B en 71.4 cs 61.0 50 100 Tool calling Structured output Summarization Classification Knowledge Instructions ministral-3-3b 3.8B en 71.8 cs 65.3 50 100 Tool calling Structured output Summarization Classification Knowledge Instructions nemotron-3-nano-4b 4.0B en 76.4 cs 62.7 50 100 Tool calling Structured output Summarization Classification Knowledge Instructions models side by side model size Tool calling Structured output Summarization Classification Knowledge Instruction following mean en cs en cs en cs en cs en cs en cs en cs cs − en functiongemma-270m0.27B 9.4 5.0 40.7 20.3 3.7 4.0 6.9 0.9 20.8 10.2 21.4 17.1 17.1 9.6 -7.5 granite-4.0-h-350m0.34B 50.6 32.7 94.1 78.8 21.2 1.0 53.8 4.9 30.5 22.0 64.1 36.5 52.4 29.3 -23.1 lfm2.5-350m0.35B 36.8 16.4 71.3 47.8 15.6 1.0 70.0 0.9 42.2 29.2 77.8 26.5 52.3 20.3 -32.0 qwen3.5-0.8b0.87B 45.6 27.4 94.7 55.9 53.0 38.5 86.0 88.7 47.0 35.0 60.5 38.9 64.5 47.4 -17.1 lfm2.5-1.2b1.2B 34.0 9.7 74.3 63.1 45.7 1.5 74.1 17.6 53.2 26.8 86.0 38.7 61.2 26.2 -35.0 qwen3.5-2b2.3B 67.3 51.3 95.7 95.5 72.0 62.8 85.6 87.6 63.8 47.8 71.3 51.7 75.9 66.1 -9.8 gemma-4-e2b2B eff. (5.1B) 62.0 54.7 95.2 95.0 80.3 79.3 94.0 92.3 58.2 54.2 85.0 69.8 79.1 74.2 -4.9 smollm3-3b3.1B 56.6 51.3 90.8 85.9 64.5 24.5 90.9 70.6 65.5 40.0 74.8 48.0 73.9 53.4 -20.5 tiny-aya-global3.3B 49.1 43.7 90.0 92.3 48.6 49.8 89.5 92.0 59.8 55.0 84.9 74.5 70.3 67.9 -2.4 granite-4.2-3b3.7B 33.0 19.8 96.3 96.2 60.7 50.8 91.3 89.4 65.2 46.8 82.0 63.3 71.4 61.0 -10.4 ministral-3-3b3.8B 64.5 57.2 78.5 60.2 60.6 69.5 92.4 91.1 70.8 57.8 64.0 55.7 71.8 65.3 -6.5 nemotron-3-nano-4b4.0B 54.7 37.1 97.3 96.6 58.1 32.3 92.4 92.9 67.2 49.2 88.6 67.9 76.4 62.7 -13.7
mean over the six skills Mean score over the six skills Czech English 0 25 50 75 100 score (0–100) functiongemma-270m 0.27B cs 9.6 en 17.1 granite-4.0-h-350m 0.34B cs 29.3 en 52.4 lfm2.5-350m 0.35B cs 20.3 en 52.3 qwen3.5-0.8b 0.87B cs 47.4 en 64.5 lfm2.5-1.2b 1.2B cs 26.2 en 61.2 qwen3.5-2b 2.3B cs 66.1 en 75.9 gemma-4-e2b 2B eff. (5.1B) cs 74.2 en 79.1 smollm3-3b 3.1B cs 53.4 en 73.9 tiny-aya-global 3.3B cs 67.9 en 70.3 granite-4.2-3b 3.7B cs 61.0 en 71.4 ministral-3-3b 3.8B cs 65.3 en 71.8 nemotron-3-nano-4b 4.0B cs 62.7 en 76.4 how it was run
Models. Instruction-tuned BF16 checkpoints, served with vLLM 0.31, greedy decoding, the checkpoint's own chat template with enable_thinking=False, and its native tool-call parser.Prompts. Every input is fully in the test language; tool descriptions and JSON schemas stay in English. The Czech json-mode set and the Czech summarization prompt are Kimi K3 translations; everything else is human-made or human-checked Czech.Judge. Summaries only: Kimi K3 (e-INFRA CZ), reasoning effort high, one judge, listwise with hidden and shuffled system names. Tool calling Czech English 0 50 100 functiongemma-270m 5.0 9.4 granite-4.0-h-350m 32.7 50.6 lfm2.5-350m 16.4 36.8 qwen3.5-0.8b 27.4 45.6 lfm2.5-1.2b 9.7 34.0 qwen3.5-2b 51.3 67.3 gemma-4-e2b 54.7 62.0 smollm3-3b 51.3 56.6 tiny-aya-global 43.7 49.1 granite-4.2-3b 19.8 33.0 ministral-3-3b 57.2 64.5 nemotron-3-nano-4b 37.1 54.7 model accuracy % via JSON text errors en cs en cs en cs functiongemma-270m9.4 5.0 0.0 0.0 0 0 granite-4.0-h-350m50.6 32.7 0.0 0.0 0 0 lfm2.5-350m36.8 16.4 0.0 0.0 0 0 qwen3.5-0.8b45.6 27.4 0.0 0.0 0 0 lfm2.5-1.2b34.0 9.7 0.0 0.6 0 0 qwen3.5-2b67.3 51.3 0.0 0.0 0 0 gemma-4-e2b62.0 54.7 0.0 0.0 0 0 smollm3-3b56.6 51.3 7.2 14.8 0 0 tiny-aya-global49.1 43.7 100.0 99.7 0 0 granite-4.2-3b33.0 19.8 3.5 11.3 0 0 ministral-3-3b64.5 57.2 0.0 0.3 0 0 nemotron-3-nano-4b54.7 37.1 0.0 0.0 0 0
↓ samples below: every input with each model's answer samples
Structured output Czech English 0 50 100 functiongemma-270m 20.3 40.7 granite-4.0-h-350m 78.8 94.1 lfm2.5-350m 47.8 71.3 qwen3.5-0.8b 55.9 94.7 lfm2.5-1.2b 63.1 74.3 qwen3.5-2b 95.5 95.7 gemma-4-e2b 95.0 95.2 smollm3-3b 85.9 90.8 tiny-aya-global 92.3 90.0 granite-4.2-3b 96.2 96.3 ministral-3-3b 60.2 78.5 nemotron-3-nano-4b 96.6 97.3 model score valid JSON schema fields en cs en cs en cs en cs functiongemma-270m40.7 20.3 53.0 40.0 35.0 13.0 34.2 7.9 granite-4.0-h-350m94.1 78.8 98.0 86.0 97.0 82.0 87.2 68.5 lfm2.5-350m71.3 47.8 99.0 98.0 59.0 25.0 55.8 20.3 qwen3.5-0.8b94.7 55.9 100.0 100.0 97.0 38.0 87.2 29.7 lfm2.5-1.2b74.3 63.1 99.0 100.0 65.0 51.0 59.0 38.3 qwen3.5-2b95.7 95.5 99.0 100.0 98.0 98.0 90.0 88.6 gemma-4-e2b95.2 95.0 100.0 100.0 96.0 96.0 89.5 89.1 smollm3-3b90.8 85.9 100.0 100.0 88.0 81.0 84.4 76.7 tiny-aya-global90.0 92.3 99.0 100.0 88.0 92.0 83.1 85.0 granite-4.2-3b96.3 96.2 100.0 100.0 98.0 100.0 90.8 88.6 ministral-3-3b78.5 60.2 98.0 95.0 53.0 25.0 84.6 60.6 nemotron-3-nano-4b97.3 96.6 100.0 100.0 99.0 100.0 92.8 89.9
↓ samples below: every input with each model's answer samples
Summarization Czech English 0 50 100 functiongemma-270m 4.0 3.7 granite-4.0-h-350m 1.0 21.2 lfm2.5-350m 1.0 15.6 qwen3.5-0.8b 38.5 53.0 lfm2.5-1.2b 1.5 45.7 qwen3.5-2b 62.8 72.0 gemma-4-e2b 79.3 80.3 smollm3-3b 24.5 64.5 tiny-aya-global 49.8 48.6 granite-4.2-3b 50.8 60.7 ministral-3-3b 69.5 60.6 nemotron-3-nano-4b 32.3 58.1 EUR-Lex (human) 81.0 71.4 model score inform. faithf. coher. concise right lang. en cs en cs en cs en cs en cs en cs functiongemma-270m3.7 4.0 1.06 1.02 1.38 1.62 2.56 1.80 1.48 1.58 96.0 86.0 granite-4.0-h-350m21.2 1.0 1.18 1.08 4.76 1.67 4.52 1.83 4.56 1.50 100.0 14.0 lfm2.5-350m15.6 1.0 1.94 1.02 1.46 1.10 2.56 1.06 2.22 1.22 100.0 88.0 qwen3.5-0.8b53.0 38.5 3.34 2.24 3.28 3.28 3.28 2.40 2.38 1.62 100.0 100.0 lfm2.5-1.2b45.7 1.5 2.68 1.04 3.20 1.10 4.48 1.22 4.54 1.12 100.0 100.0 qwen3.5-2b72.0 62.8 4.36 3.54 3.60 3.72 4.60 3.50 3.64 2.78 100.0 100.0 gemma-4-e2b80.3 79.3 4.08 4.26 4.44 4.18 4.96 4.28 4.16 3.74 100.0 100.0 smollm3-3b64.5 24.5 3.32 2.00 4.04 2.54 3.94 2.20 3.08 1.58 100.0 78.0 tiny-aya-global48.6 49.8 3.16 3.14 2.86 2.94 4.44 4.58 3.72 4.38 100.0 100.0 granite-4.2-3b60.7 50.8 3.88 3.44 3.18 2.76 3.52 3.18 2.70 3.10 100.0 100.0 ministral-3-3b60.6 69.5 4.56 4.46 2.82 3.32 4.82 4.54 3.74 3.88 100.0 100.0 nemotron-3-nano-4b58.1 32.3 3.70 2.38 3.12 2.50 4.28 2.08 4.12 2.00 100.0 100.0 human71.4 81.0 3.78 4.26 4.05 4.29 4.16 4.26 3.43 3.65 100.0 100.0
↓ samples below: every input with each model's answer samples
Classification Czech English 0 50 100 functiongemma-270m 0.9 6.9 granite-4.0-h-350m 4.9 53.8 lfm2.5-350m 0.9 70.0 qwen3.5-0.8b 88.7 86.0 lfm2.5-1.2b 17.6 74.1 qwen3.5-2b 87.6 85.6 gemma-4-e2b 92.3 94.0 smollm3-3b 70.6 90.9 tiny-aya-global 92.0 89.5 granite-4.2-3b 89.4 91.3 ministral-3-3b 91.1 92.4 nemotron-3-nano-4b 92.9 92.4 model accuracy hit limit en cs en cs functiongemma-270m6.9 0.9 2 143 granite-4.0-h-350m53.8 4.9 0 0 lfm2.5-350m70.0 0.9 0 1 qwen3.5-0.8b86.0 88.7 0 0 lfm2.5-1.2b74.1 17.6 0 13 qwen3.5-2b85.6 87.6 0 0 gemma-4-e2b94.0 92.3 0 0 smollm3-3b90.9 70.6 0 0 tiny-aya-global89.5 92.0 0 0 granite-4.2-3b91.3 89.4 5 0 ministral-3-3b92.4 91.1 0 1 nemotron-3-nano-4b92.4 92.9 0 0
↓ samples below: every input with each model's answer samples
Knowledge Czech English 0 50 100 functiongemma-270m 10.2 20.8 granite-4.0-h-350m 22.0 30.5 lfm2.5-350m 29.2 42.2 qwen3.5-0.8b 35.0 47.0 lfm2.5-1.2b 26.8 53.2 qwen3.5-2b 47.8 63.8 gemma-4-e2b 54.2 58.2 smollm3-3b 40.0 65.5 tiny-aya-global 55.0 59.8 granite-4.2-3b 46.8 65.2 ministral-3-3b 57.8 70.8 nemotron-3-nano-4b 49.2 67.2 model accuracy agnostic sensitive en cs en cs en cs functiongemma-270m20.8 10.2 20.5 12.5 21.0 8.0 granite-4.0-h-350m30.5 22.0 32.5 22.5 28.5 21.5 lfm2.5-350m42.2 29.2 46.5 30.0 38.0 28.5 qwen3.5-0.8b47.0 35.0 54.5 39.0 39.5 31.0 lfm2.5-1.2b53.2 26.8 57.0 28.0 49.5 25.5 qwen3.5-2b63.8 47.8 68.0 53.5 59.5 42.0 gemma-4-e2b58.2 54.2 60.5 57.0 56.0 51.5 smollm3-3b65.5 40.0 67.5 40.5 63.5 39.5 tiny-aya-global59.8 55.0 64.5 58.0 55.0 52.0 granite-4.2-3b65.2 46.8 71.0 51.5 59.5 42.0 ministral-3-3b70.8 57.8 73.5 59.5 68.0 56.0 nemotron-3-nano-4b67.2 49.2 71.0 51.5 63.5 47.0
↓ samples below: every input with each model's answer samples
Instruction following Czech English 0 50 100 functiongemma-270m 17.1 21.4 granite-4.0-h-350m 36.5 64.1 lfm2.5-350m 26.5 77.8 qwen3.5-0.8b 38.9 60.5 lfm2.5-1.2b 38.7 86.0 qwen3.5-2b 51.7 71.3 gemma-4-e2b 69.8 85.0 smollm3-3b 48.0 74.8 tiny-aya-global 74.5 84.9 granite-4.2-3b 63.3 82.0 ministral-3-3b 55.7 64.0 nemotron-3-nano-4b 67.9 88.6 model prompt strict inst. strict prompt loose inst. loose en cs en cs en cs en cs functiongemma-270m16.3 13.1 26.4 21.2 17.2 14.2 27.0 22.6 granite-4.0-h-350m59.2 31.7 69.0 41.2 61.5 34.3 70.8 44.7 lfm2.5-350m74.1 21.4 81.5 31.6 76.0 23.3 82.9 33.8 qwen3.5-0.8b56.6 33.8 64.5 44.1 61.5 37.5 69.7 47.7 lfm2.5-1.2b83.9 33.3 88.2 44.1 85.5 36.4 89.6 47.2 qwen3.5-2b67.6 46.6 75.0 56.9 73.2 51.0 79.8 60.8 gemma-4-e2b82.8 65.7 87.2 73.9 84.6 70.6 88.9 77.6 smollm3-3b71.3 42.4 78.2 53.6 76.2 45.9 82.6 57.0 tiny-aya-global82.5 70.9 87.2 78.2 85.1 75.3 89.6 81.7 granite-4.2-3b79.0 59.0 85.0 67.6 81.8 64.6 87.2 72.0 ministral-3-3b58.5 50.8 69.5 60.7 66.7 57.8 76.1 67.1 nemotron-3-nano-4b86.9 63.4 90.2 72.4 88.8 66.2 91.6 74.4
↓ samples below: every input with each model's answer samples
