lokalni.ai lokalni.ai ⁄

Do LLMs know the Czech vyjmenovaná slova?

15 models list the vyjmenovaná slova as JSON, scored against the 99 items outside brackets in ÚJČ's Přehled vyjmenovaných slov.
best open-weight0.83Muse Spark Open · on · 1 run
best over 5 runs0.76DeepSeek V4.1 Flash · on · ± 0.06
best closed0.93Gemini 4 Argon · on · 1 run
settings with F1 ≥ 0.56/2415 models · 96 runs
gold items9984 words + 15 names, ÚJČ

Setup & prompt

15 models, 24 settings
e-INFRA, local vLLM, OpenRouter
modelcheckpoint / API slugran onweightsmodesreasoning + answer tokensruns
Kimi K3.1-Promoonshotai/Kimi-K3.1-Proe-INFRAas servedon, offon 32k + 16k (native), off 16.0kon 5, off 5
DeepSeek V4.1 Flashdeepseek-ai/DeepSeek-V4.1-Flash-1017e-INFRAas servedon, offon 32k + 16k (native), off 16.0kon 5, off 5
GPT-OSS-120Bopenai/gpt-oss-120be-INFRAMXFP4onon 32k + 16k (forced)on 5
Qwen3.8-Flash-NextQwen/Qwen3.8-Flash-Next-FP8e-INFRAFP8on, offon 32k + 16k (forced), off 16.0kon 5, off 5
Qwen3.8-27BQwen/Qwen3.8-27B-FP8e-INFRAFP8on, offon 32k + 16k (forced), off 16.0kon 5, off 5
Gemma-4-31Bgoogle/gemma-4-31B-ite-INFRAas servedon, offon 32k + 16k (forced), off 16.0kon 5, off 5
Qwen3.6-35B-A3BQwen/Qwen3.6-35B-A3B-FP8local vLLMFP8on, offon 32k + 16k (native), off 16.0kon 5, off 5
Gemma-4-26B-A4BRedHatAI/gemma-4-26B-A4B-it-FP8-dynamiclocal vLLMFP8on, offon 32k + 16k (native), off 16.0kon 5, off 5
Gemma-4-E4Bgoogle/gemma-4-E4B-itlocal vLLMFP8 (on load)on, offon 32k + 16k (native), off 16.0kon 5, off 5
Tiny Aya GlobalCohereLabs/tiny-aya-globallocal vLLMFP8 (on load)offoff 7.2koff 5
GPT-6.1 Terraopenai/gpt-6.1-terraOpenRouterclosedon, offon effort high, 48.0k in total, off 16.0kon 1, off 1
Claude Opus 5.5anthropic/claude-opus-5.5OpenRouterclosedonon effort high, 48.0k in totalon 1
Gemini 4 Argongoogle/gemini-4-argonOpenRouterclosedonon effort high, 48.0k in totalon 1
Mistral Large 4mistralai/mistral-large-4-0OpenRouteras served (Mistral)onon effort high, 48.0k in totalon 1
Muse Spark Openmeta/muse-spark-openOpenRouterFP4 (BaseTen)onon effort high, 48.0k in totalon 1
the prompt (Czech, no system prompt)
Vypiš vyjmenovaná slova podle přehledu vyjmenovaných slov v Internetové jazykové příručce Ústavu pro jazyk český AV ČR (přehled vychází z akademického vydání Pravidel českého pravopisu z roku 1993).

- Uveď jen slova, která přehled uvádí jako vyjmenovaná. Slova příbuzná a odvozená, která přehled uvádí v závorkách za vyjmenovaným slovem, neuváděj.
- Zahrň i vlastní jména (zeměpisná a osobní), která přehled uvádí.
- Slova piš v takovém tvaru, v jakém jsou v přehledu.
- Odpověz pouze JSON objektem. Klíče jsou písmena (malými písmeny), hodnoty jsou seznamy slov pro dané písmeno: {"písmeno": ["slovo", "slovo", ...], ...}. Nic dalšího nepiš.

Computational resources were provided by the e-INFRA CZ project (ID:90254), supported by the Ministry of Education, Youth and Sports of the Czech Republic.

Leaderboard

reasoning off
hollow
reasoning on
filled; high effort where the model has levels
dimmed
closed weights
whisker
± 1 sd over the 5 runs (none for single-run models)
reas. tok
median, reasoning on
0.00.20.40.60.81.0
off
on
Δ
reas. tok
OpenRouter · closed · 1 run
–
0.93
–
12.3k
OpenRouter · closed · 1 run
0.80
0.84
+0.04
5.9k
OpenRouter · FP4 (BaseTen) · 1 run
–
0.83
–
8.0k
OpenRouter · closed · 1 run
–
0.82
–
2.5k
e-INFRA · as served
0.40
0.76
+0.37
32.0k
OpenRouter · as served (Mistral) · 1 run
–
0.36
–
18.8k
e-INFRA · as served
0.10
0.30
+0.20
12.4k
e-INFRA · MXFP4
–
0.19
–
32.0k
e-INFRA · FP8
0.01
0.09
+0.08
32.0k
e-INFRA · as served
0.00
0.03
+0.02
10.3k
local vLLM · FP8 (on load)
0.00
0.02
+0.02
1.0k
local vLLM · FP8
0.00
0.01
+0.01
20.6k
e-INFRA · FP8
0.01
0.00
0.00
32.0k
local vLLM · FP8
0.00
0.00
0.00
13.3k
local vLLM · FP8 (on load)
0.00
–
–
–

What reasoning costs

Each dot is one reasoning-on run, rings are model means; reasoning tokens on a log scale.

0.000.250.500.751.001.0k3.0k10.0k32.0kcapreasoning tokens (log scale)F1Qwen3.8-Flash-NextGPT-OSS-120BKimi K3.1-ProMistral Large 4DeepSeek V4.1 FlashClaude Opus 5.5Muse Spark OpenGPT-6.1 TerraGemini 4 Argon
0.000.250.500.751.001.0k3.0k10.0k32.0kcapreasoning tokens (log scale)F1Qwen3.8-Flash-NextGPT-OSS-120BKimi K3.1-ProMistral Large 4DeepSeek V4.1 FlashClaude Opus 5.5Muse Spark OpenGPT-6.1 TerraGemini 4 Argon
run
one reasoning-on run
model mean
geometric mean of tokens, mean F1
dimmed
closed weights
forced
reasoning cut at the 32k cap, then made to answer
×no answer
no parseable JSON (F1 = 0)

Unlabelled rings near F1 = 0 (hover a dot for its model): Gemma-4-26B-A4B, Gemma-4-31B, Gemma-4-E4B, Qwen3.6-35B-A3B, Qwen3.8-27B.

What each model knows

Recall per letter and for the proper names (mean over runs), or every gold item against every setting.

b12l17m19p18s13v16z4names15
on1.001.001.000.940.850.811.000.80
on0.830.760.740.780.850.691.000.40
off0.580.760.740.670.850.691.000.20
on0.580.650.740.940.850.751.000.40
on0.580.760.740.720.850.751.000.20
on0.720.800.750.770.740.600.750.23
off0.570.460.520.430.650.420.050.21
on0.170.350.370.170.380.190.250.00
on0.250.150.310.330.480.140.000.01
off0.180.080.130.100.230.070.000.00
on0.180.110.140.130.260.060.000.00
on0.120.050.040.120.030.070.000.05
off0.020.000.010.000.020.000.000.00
on0.020.000.030.020.030.000.000.00
off0.000.000.000.010.000.000.000.00
on0.000.000.020.010.030.010.000.00
off0.000.000.010.000.000.000.000.00
on0.000.000.020.000.000.000.000.00
off0.000.000.000.000.000.000.000.00
on0.000.000.000.000.020.000.000.00
off0.000.000.000.000.030.000.000.00
on0.000.000.000.000.000.000.000.00
off0.020.000.000.000.000.000.000.00
off0.000.000.000.000.000.000.000.00

Most common wrong words

Listed words that aren't gold items, ranked by how many settings produced them (top 80).

derived
derived word (in brackets in ÚJČ)
near
one edit from a gold word
misplaced
gold word under the wrong letter
other
not a listed word
wordkeylabelsettingsrunslisted by
bytbderived1222Claude Opus 5.5 on, DeepSeek V4.1 Flash on, GPT-6.1 Terra off, GPT-6.1 Terra on, GPT-OSS-120B on, Gemini 4 Argon on, Kimi K3.1-Pro off, Kimi K3.1-Pro on, Mistral Large 4 on, Muse Spark Open on, Qwen3.8-27B on, Qwen3.8-Flash-Next on
bydlitbderived1220Claude Opus 5.5 on, DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, GPT-6.1 Terra off, GPT-6.1 Terra on, GPT-OSS-120B on, Gemini 4 Argon on, Kimi K3.1-Pro off, Kimi K3.1-Pro on, Mistral Large 4 on, Muse Spark Open on, Qwen3.8-Flash-Next on
obyvatelbderived816Claude Opus 5.5 on, DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, GPT-6.1 Terra off, GPT-6.1 Terra on, Gemini 4 Argon on, Mistral Large 4 on, Muse Spark Open on
příbytekbderived812Claude Opus 5.5 on, DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, GPT-6.1 Terra off, GPT-6.1 Terra on, Gemini 4 Argon on, Mistral Large 4 on, Muse Spark Open on
fyzikafother712DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, GPT-OSS-120B on, Gemma-4-31B off, Kimi K3.1-Pro off, Kimi K3.1-Pro on, Qwen3.8-27B off
dobytekbderived711Claude Opus 5.5 on, DeepSeek V4.1 Flash on, GPT-6.1 Terra off, GPT-6.1 Terra on, Gemini 4 Argon on, Mistral Large 4 on, Muse Spark Open on
nábytekbderived711Claude Opus 5.5 on, DeepSeek V4.1 Flash on, GPT-6.1 Terra off, GPT-6.1 Terra on, Gemini 4 Argon on, Mistral Large 4 on, Muse Spark Open on
mýdlomderived612DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, GPT-OSS-120B on, Gemma-4-E4B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on
pyšnýpderived611DeepSeek V4.1 Flash off, GPT-OSS-120B on, Gemma-4-E4B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on, Qwen3.8-Flash-Next on
lysálderived69DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, GPT-OSS-120B on, Gemma-4-26B-A4B on, Kimi K3.1-Pro on, Qwen3.8-Flash-Next on
bůhbother68Gemma-4-26B-A4B off, Gemma-4-26B-A4B on, Gemma-4-31B on, Kimi K3.1-Pro on, Qwen3.8-27B on, Qwen3.8-Flash-Next on
bydletbderived511DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, Kimi K3.1-Pro off, Kimi K3.1-Pro on, Qwen3.8-Flash-Next on
dýmdother510GPT-OSS-120B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on, Qwen3.8-27B on, Qwen3.8-Flash-Next on
pyžamopother58DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, GPT-OSS-120B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on
gymnáziumgother57DeepSeek V4.1 Flash off, GPT-OSS-120B on, Kimi K3.1-Pro off, Qwen3.8-27B off, Qwen3.8-27B on
hájhother56Gemma-4-26B-A4B off, Gemma-4-26B-A4B on, Gemma-4-31B on, Gemma-4-E4B off, Qwen3.8-27B on
kůňkother56Gemma-4-26B-A4B off, Gemma-4-31B off, Gemma-4-31B on, Qwen3.8-27B off, Qwen3.8-27B on
rýčrother55DeepSeek V4.1 Flash on, Kimi K3.1-Pro off, Kimi K3.1-Pro on, Mistral Large 4 on, Qwen3.6-35B-A3B on
čepýřitpderived55Claude Opus 5.5 on, DeepSeek V4.1 Flash off, GPT-6.1 Terra off, GPT-6.1 Terra on, Gemini 4 Argon on
sýriesother410DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, Kimi K3.1-Pro off, Kimi K3.1-Pro on
mýtusmother49GPT-OSS-120B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on, Qwen3.8-Flash-Next on
myslmderived48DeepSeek V4.1 Flash off, GPT-OSS-120B on, Gemma-4-E4B on, Kimi K3.1-Pro on
dýchatdnear47GPT-OSS-120B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on, Qwen3.6-35B-A3B off
zvykzderived47DeepSeek V4.1 Flash off, GPT-OSS-120B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on
bylbnear46DeepSeek V4.1 Flash off, Kimi K3.1-Pro off, Qwen3.8-27B off, Qwen3.8-Flash-Next off
bystřicebderived46DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, Kimi K3.1-Pro on, Qwen3.8-Flash-Next on
běhbother46Gemma-4-31B off, Gemma-4-31B on, Gemma-4-E4B off, Gemma-4-E4B on
cypřišcother46DeepSeek V4.1 Flash off, GPT-OSS-120B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on
lyžovatlderived46DeepSeek V4.1 Flash off, GPT-OSS-120B on, Kimi K3.1-Pro off, Mistral Large 4 on
nůžnother46Gemma-4-31B off, Gemma-4-31B on, Gemma-4-E4B on, Qwen3.8-27B on
rybarother46GPT-OSS-120B on, Kimi K3.1-Pro off, Mistral Large 4 on, Qwen3.8-27B off
rádrother46Gemma-4-E4B off, Gemma-4-E4B on, Qwen3.6-35B-A3B off, Qwen3.6-35B-A3B on
rýmarother46GPT-OSS-120B on, Gemma-4-31B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on
synonymumsother46DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, Kimi K3.1-Pro off, Kimi K3.1-Pro on
systémsother46DeepSeek V4.1 Flash off, GPT-OSS-120B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on
bývalýbderived45DeepSeek V4.1 Flash off, Gemma-4-31B off, Gemma-4-31B on, Kimi K3.1-Pro off
cizineccother45Qwen3.6-35B-A3B on, Qwen3.8-27B off, Qwen3.8-27B on, Tiny Aya Global off
cizícother45Gemma-4-26B-A4B on, Qwen3.6-35B-A3B on, Qwen3.8-27B on, Qwen3.8-Flash-Next on
cykluscother45DeepSeek V4.1 Flash off, GPT-OSS-120B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on
gymnastikagother45DeepSeek V4.1 Flash off, GPT-OSS-120B on, Kimi K3.1-Pro off, Qwen3.8-27B off
mámamother45Gemma-4-31B on, Gemma-4-E4B off, Qwen3.6-35B-A3B on, Qwen3.8-Flash-Next on
nýtnnear45GPT-OSS-120B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on, Qwen3.6-35B-A3B off
rýžerother45GPT-OSS-120B on, Gemma-4-31B off, Gemma-4-31B on, Kimi K3.1-Pro off
týdentother45GPT-OSS-120B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on, Tiny Aya Global off
břehbother44Gemma-4-31B on, Gemma-4-E4B on, Kimi K3.1-Pro on, Qwen3.6-35B-A3B on
kyjknear44Kimi K3.1-Pro off, Kimi K3.1-Pro on, Qwen3.8-Flash-Next off, Qwen3.8-Flash-Next on
lítlother44Gemma-4-31B off, Gemma-4-E4B off, Qwen3.6-35B-A3B off, Qwen3.6-35B-A3B on
pádpother44Gemma-4-31B off, Gemma-4-E4B off, Gemma-4-E4B on, Qwen3.8-Flash-Next on
sůlsother44Gemma-4-31B off, Gemma-4-31B on, Gemma-4-E4B on, Qwen3.8-27B on
vysoké mýtovother44DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, Kimi K3.1-Pro on, Mistral Large 4 on
babičkabother39Gemma-4-31B off, Gemma-4-31B on, Qwen3.8-Flash-Next on
hřbethother37Gemma-4-31B off, Gemma-4-31B on, Gemma-4-E4B on
mývalmderived37DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, Kimi K3.1-Pro on
sytitsother37DeepSeek V4.1 Flash off, Kimi K3.1-Pro off, Kimi K3.1-Pro on
batohbother36Gemma-4-31B off, Gemma-4-31B on, Qwen3.8-Flash-Next on
hymnahother36DeepSeek V4.1 Flash off, GPT-OSS-120B on, Kimi K3.1-Pro off
lýdielother36DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, Kimi K3.1-Pro on
pytlákpderived36DeepSeek V4.1 Flash off, Kimi K3.1-Pro off, Kimi K3.1-Pro on
cyrilcother35DeepSeek V4.1 Flash off, GPT-OSS-120B on, Qwen3.8-Flash-Next on
hadhother35Gemma-4-31B off, Qwen3.6-35B-A3B on, Qwen3.8-Flash-Next on
hýbathother35GPT-OSS-120B on, Kimi K3.1-Pro off, Qwen3.6-35B-A3B off
kyselýkother35GPT-OSS-120B on, Kimi K3.1-Pro off, Qwen3.8-Flash-Next on
lyralother35DeepSeek V4.1 Flash off, GPT-OSS-120B on, Kimi K3.1-Pro on
lyrikalother35DeepSeek V4.1 Flash off, Kimi K3.1-Pro off, Kimi K3.1-Pro on
vyšehradvderived35DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, Qwen3.8-Flash-Next on
výživavother35DeepSeek V4.1 Flash off, GPT-OSS-120B on, Kimi K3.1-Pro off
zývatzother35DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, GPT-OSS-120B on
bytostbderived34Kimi K3.1-Pro off, Kimi K3.1-Pro on, Qwen3.8-Flash-Next on
bílýbother34Gemma-4-31B off, Gemma-4-31B on, Qwen3.8-27B on
chytrýcother34GPT-OSS-120B on, Gemma-4-26B-A4B on, Kimi K3.1-Pro on
cyklistacother34DeepSeek V4.1 Flash off, Kimi K3.1-Pro off, Kimi K3.1-Pro on
cynikcother34GPT-OSS-120B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on
dýkadother34GPT-OSS-120B on, Kimi K3.1-Pro off, Qwen3.8-27B on
fialkafother34Gemma-4-31B off, Gemma-4-31B on, Qwen3.8-Flash-Next on
myjavamother34DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, Kimi K3.1-Pro on
rysrnear34GPT-OSS-120B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on
rýtrnear34DeepSeek V4.1 Flash on, GPT-OSS-120B on, Kimi K3.1-Pro on
týkattother34GPT-OSS-120B on, Kimi K3.1-Pro off, Kimi K3.1-Pro on
zvykvderived34DeepSeek V4.1 Flash off, DeepSeek V4.1 Flash on, Mistral Large 4 on
abecedaaother33Gemma-4-E4B on, Qwen3.6-35B-A3B on, Qwen3.8-27B off

Every run

Pick a run to open its answer, the gold items it missed and its reasoning trace (← → to step).