Local AI Tested: The German Model Loses at German
Many companies want to use AI but do not want to hand their data to the cloud. What can models do that run entirely on your own hardware? And which one should you pick for German text?
I tested four current open-weight models: three popular models from China and one from Germany (Kolibri-1). The hardware is two consumer graphics cards (RTX 5090 and RTX 3090) in a consumer system with 128 GB of RAM.
The tasks
Each model solved eight real-world tasks, scored 0 to 10 points each against fixed criteria:
- check invoices against purchase orders, price lists and billing rules
- produce a monthly statement by hand, including credit notes and special prices
- find and fix bugs in existing code
- implement appointment scheduling from a specification (time zones, daylight saving time)
- build a web application (an invoice editor)
- proofread a German notice, check it for factual errors and rewrite it in Plain Language (Einfache Sprache)
- solve hard algorithm problems and answer a knowledge quiz
The result
Out of a maximum of 80 points:
- DeepSeek-V4.1-Flash, 2.25-bit quantisation: 72
- Qwen3.8-27B, UD-Q6_K_XL: 68.5
- Qwen3.8-Flash-Next, UD-IQ4_XS: 68.5
- Kolibri-1, Q4_K_M: 59
The two Qwen models share second place.
The Chinese models are better at German
Of all things, on the German text task the German model came in behind all three Chinese ones. The task asked for more than spelling: the notice had to be corrected, checked for factual errors and rewritten in Plain Language. Qwen3.8-Flash-Next scored 9.5 points, DeepSeek-V4.1-Flash 8.5, Qwen3.8-27B 6.5 and Kolibri-1 only 6.
So a model from Germany is not automatically the best choice for German text. This is no verdict on German models in general; I tested only one. But if you use AI for correspondence or specialist texts, choose the model for that task rather than by its origin, and proofread the output.
Precise specifications separate the models
The scores spread even further on the scheduler built to specification: from 3 to 10 points. Qwen3.8-27B implemented it flawlessly, while the fast Qwen3.8-Flash-Next managed only 3. Time zones and daylight saving time are exactly the details where a model either works cleanly or does not.
All four can act as agents
Every model handled the invoice check and the monthly statement (almost) flawlessly. They read files on their own, applied rules, spotted the traps in the data and turned down unjustified customer claims with reasons. Workflows like these can run locally today.
Fast does not mean bad
Qwen3.8-Flash-Next generates 148 tokens per second and reaches the same score as the 27B model. The full test run took 56 minutes instead of 135. It is also my preferred model for everyday work.
More thinking did not buy a better score either. Qwen3.8-27B generated around 535,000 tokens for all eight tasks, the most in the test. DeepSeek-V4.1-Flash got by with 369,000 and still scored the most points.
Heavily compressed models can still do a lot
DeepSeek-V4.1-Flash, shrunk to 2.25 bits per weight, scores the most points but is the slowest model in the test. That is down to its size: 552 billion weights.
Conclusion
Local AI is ready for many workflows in accounting, back-office work and software development, on hardware that fits on a desk. What matters is choosing the right model for each task and testing it against your own workflows first. That includes language: only a test shows which model writes good German.
Thinking about running AI in-house without sending data to the cloud? Get in touch, I am happy to share what I have learned. How such a project runs is described under AI on your own hardware.