CzechVoice: Czech speech to text on the device
Czech speech to text on the device · weights free for non-commercial use
Whisper large-v3-turbo fine-tuned on Czech. It transcribes Czech more accurately than the open models we tried, and its 452 MiB 4-bit file runs offline on an ordinary phone: 6.9 seconds of speech in 2.7 seconds on a Pixel 9a, on the processor alone. The audio never leaves the device's memory.
Main results
| model | parliament | FLEURS | noise | numbers right |
|---|---|---|---|---|
| CzechVoice | 1.8 % | 7.3 % | 6.5 % | 99.3 % |
| NVIDIA Canary-1b-v2 | 7.5 % | 8.4 % | 22.4 % | 94.8 % |
| Whisper large-v3-turbo (our starting point) | 9.5 % | 11.4 % | 29.5 % | 96.5 % |
| NVIDIA Parakeet-TDT-0.6b-v3 | 10.3 % | 11.9 % | 25.8 % | 94.6 % |
Parliament: ParCzech4Speech test split, 729 segments; FLEURS: Czech test set, 723 sentences; noise: the parliamentary segments with reverberation and crowd babble at 10 dB. Parliamentary speech is the training domain of CzechVoice, so FLEURS is the least favourable comparison.
Licence of the weights: free for non-commercial use; commercial use: info@moravian.ai.
ThinCtx: language models with a small context memory
Language models with a small context memory · PolyForm Small Business licence
ThinCtx-small: the quality of Qwen3.5 in a small GPU memory. Qwen3.5 0.8B, 2B and 4B, converted so that the GPU memory for the context stays small and grows about 15 to 20 times more slowly than the original's cache, with the weights stored at 5.4 to 6.4 bits per weight and the context kept compressed in system memory. At 128k tokens the 4B-small needs 4.0 GB of GPU memory instead of 12.0 GB for the original, and at 540k tokens 4.7 GB instead of 24.5 GB, so a 24 GB card runs contexts the original does not fit. The thinctx 1.0 runtime also shares the memory of a common system prompt across conversations. Full-precision versions are available too.
--compress_context runs the full-precision model with the compressed context store; ThinCtx-small also quantizes the weights.Quickstart
curl -LsSf https://astral.sh/uv/install.sh | sh && source $HOME/.local/bin/env # only if uv is missing
uv venv --managed-python --python 3.12 ~/thinctx && source ~/thinctx/bin/activate
uv pip install thinctx
thinctx serve moravianai/ThinCtx-Qwen3.5-2B-small --port 8000 --auth user:pass # OpenAI-compatible server
thinctx chat moravianai/ThinCtx-Qwen3.5-2B-small # chat in the terminalIn Python: thinctx.load("moravianai/ThinCtx-Qwen3.5-2B-small"). Linux x86_64 or Windows through WSL2, with an NVIDIA GPU; Triton needs gcc on the first start. On Windows: wsl --install -d Ubuntu-24.04 in an admin PowerShell, then the same steps in Ubuntu. Details on each model card.
Main results
| model | |||
|---|---|---|---|
RULER: synthetic tests at 65 536 tokens: six needle-in-a-haystack variants (one or several needles, values or questions), variable tracking, common and frequent word extraction, and single-hop question answering; 10 of RULER's 13 tasks. 10 items per task, 100 in all; RULER's string-match score, higher is better. RULER, Hsieh et al. 2024 | |||
BFCL, the Berkeley Function Calling Leaderboard, non-live AST categories: simple, multiple, parallel and parallel-multiple calls written against curated function documentation, checked by BFCL's own AST checker; thinking off, greedy decoding. The number is BFCL's non-live AST score, the mean over these four categories (% correct), higher is better. Berkeley Function Calling Leaderboard | |||
LongBench v1: 16 English tasks in 6 categories (single- and multi-document QA, summarisation, few-shot learning, synthetic tasks, code), 3 750 items, with the official prompts, truncation and metrics. The number is the mean of the 16 task scores, higher is better. LongBench, Bai et al. 2023 | |||
| Qwen3.5-4B, original | 97.0 | 86.2 | 45.1 |
| ThinCtx-Qwen3.5-4B | 95.5 | 85.1 | 45.1 |
| Qwen3.5-2B, original | 95 | 68.5 | 42.6 |
| ThinCtx-Qwen3.5-2B | 89 | 70.4 | 42.5 |
BABILong: bAbI reasoning questions (tasks qa1 to qa5: one, two or three supporting facts, relations between two or three objects) whose facts are scattered through 65 536 tokens of book text. 20 questions per task, 100 in all; the share answered correctly, higher is better. BABILong, Kuratov et al. 2024 | |||
RULER: synthetic tests at 65 536 tokens: six needle-in-a-haystack variants (one or several needles, values or questions), variable tracking, common and frequent word extraction, and single-hop question answering; 10 of RULER's 13 tasks. 10 items per task, 100 in all; RULER's string-match score, higher is better. RULER, Hsieh et al. 2024 | |||
Real agent sessions of 8 192 and 32 768 tokens (16 per length) whose tool definitions or documented commands sit far back in the context, up to 32 000 tokens before the call. The share of the calls' function names, parameter names and command names predicted as in the session; higher is better. | |||
| 43 | 95 | 96.4 | |
The unchanged Qwen3.5-2B with its whole context cache on the GPU: the reference for every column. | |||
| 1 | 5 | 66.9 | |
The unchanged model keeps only the latest tokens and the first few tokens of the context in its cache, as in StreamingLLM; its GPU memory stays constant and everything in between is forgotten. StreamingLLM, Xiao et al. 2023 | |||
| 37 | 68 | 90.2 | |
The unchanged model with SentenceKV-style cache retrieval (Zhu et al. 2025), as published: the full cache is kept off the GPU and the model reads part of it at each step. The model itself is not changed. | |||
| 49 | 89 | 96.6 | |
The converted Qwen3.5-2B: a small part of the context on the GPU (growing slowly with its length), the full context in system memory. Same items and settings as the other rows. | |||
Same items and settings for every row. Scores in %. BFCL: non-live categories. Distant tool calls: tool definitions far back in the context.
Releases
| name | status |
|---|---|
| moravianai/ThinCtx-Qwen3.5-4B-small | open download on Hugging Face |
| moravianai/ThinCtx-Qwen3.5-2B-small | open download on Hugging Face |
| moravianai/ThinCtx-Qwen3.5-0.8B-small | open download on Hugging Face; not recommended for tool calling |
| moravianai/ThinCtx-Qwen3.5-4B | open download on Hugging Face |
| moravianai/ThinCtx-Qwen3.5-2B | open download on Hugging Face |
| moravianai/ThinCtx-Qwen3.5-0.8B | open download on Hugging Face; not recommended for tool calling |
| pip install thinctx | runtime with an OpenAI-compatible server; 1.0 shares memory across conversations |
Licence: PolyForm Small Business 1.0.0 (licensor Jan Rosecký): free for companies with fewer than 100 people and under USD 1 million revenue, and for anyone for personal non-commercial use, non-commercial research and education, and evaluation and testing. Production use by larger organisations needs a commercial licence: info@moravian.ai.
ThinEmbed: compressed query encoders
Compressed query encoders · open source, Apache-2.0
ThinEmbed compresses the query side of an embedding model into a small file that runs in the browser on WebGPU. The document index stays unchanged, and the query is encoded on the user's device. Code, runtime and released files are open source.
Quickstart
import { loadVqwClient } from 'https://moravian.ai/vqweb/vqweb.js'; // or: npm install @moravian/thinembed-vq
const client = await loadVqwClient(
'https://huggingface.co/moravianai/ThinEmbed-Qwen3-Embedding-0.6B-vq/resolve/main/qwen3-0.6b-vq3.5d2-webfaq_q-vocfull.vqw');
const q = await client.embed('Kolik stojí parkování v centru Prahy?'); // Float32Array(1024)
// score = dot(q, d) against your Qwen/Qwen3-Embedding-0.6B document vectorsNeeds WebGPU with shader-f16 (Chrome, Edge).
Main results
| query encoder | file | nDCG@10 | p50 per query |
|---|---|---|---|
| harrier-0.6b, full precision (reference) | 1 143 MiB (fp16) | 0.7559 | – |
| Q3_K, llama.cpp format, wllama on WebGPU | 235.1 MiB | 0.7481 | 444 ms |
| IQ2_M, llama.cpp format, wllama on WebGPU | 199.3 MiB | 0.7387 | 508 ms |
| IQ2_XS, smallest llama.cpp format, native | 177.5 MiB | 0.7097 | – |
| ThinEmbed, 2.1 bits per weight, WebGPU | 119.5 MiB | 0.7375 | 103 ms |
| ThinEmbed, 1.8 bits per weight, WebGPU | 105.3 MiB | 0.7109 | 102 ms |
nDCG@10 against the model's own fp32 document index. Browser rows: one idle laptop, Intel integrated GPU, headless Chrome. The ThinEmbed rows are the released files. –: not run in the browser. One device, one model, one corpus.
Full benchmarks and details → · Browser demo
Releases
| release | where |
|---|---|
| Code and report | GitHub: rosecky/embedding-quantization-public |
| thinembed | PyPI: thinembed |
| @moravian/thinembed-vq | npm: @moravian/thinembed-vq |
| moravianai/ThinEmbed-harrier-0.6b | ThinEmbed-harrier-0.6b · ThinEmbed-harrier-0.6b-vq |
| moravianai/ThinEmbed-Qwen3-Embedding-0.6B | ThinEmbed-Qwen3-Embedding-0.6B · ThinEmbed-Qwen3-Embedding-0.6B-vq |
| moravianai/ThinEmbed-bge-m3 | ThinEmbed-bge-m3 |
Code Apache-2.0. Client files under their base model's licence (MIT or Apache-2.0).