Try-Works

In April this year, when everyone on the internet was speculating that the last open Qwen model had been released and other labs may end open sourcing their models too, I explained why Alibaba and other Chinese labs would continue releasing open models, and also that more labs would enter the open model race including labs outside of China.

This has since been proven correct: we have received Qwen 3.8, DeepSeek V4.x, Kimi K3, GLM 5.x, MiMo models, StepFun. Nvidia has even stepped up and built their own lab through acquisition, Meta released Glimmer, and we've seen too many western labs release small models to mention; we've also seen specialized decision models like Jev released and EmbeddingGemma 2 from Google.

The reason Chinese labs continued open source their models is that they must do so as part of their GTM. Without open models, no awareness, no buzz on X or reddit, and therefore no inference providers offering their models with a revshare. The reason western labs release open weight models is the same, to stay relevant and stand out, and eventually monetize that awareness. Chinese labs have no ability to run the type of PR campaigns in tier one international media like OpenAI and Anthropic do, so they need to give away their models instead.

A second reason why labs continued and will continue to release open models is because we're in a battle of standards. Alibaba explicitly released Qwen 3.8 Flash Next so that providers would have time to get ready to support the new Engram architecture. Similarly, a person from Z.ai talked about "releasing models to set the norm":

With the recap done, let's look at what new developments can be spotted from the past three months of model activity.

Local AI is now reality

Thanks to a number of factors including the release of finally-capable models like Qwen 3.8 27B, GLM 5.3 Flash, and so on, DGX Sparks and M5 Ultras, and perhaps most importantly purpose-built inference engines like Tensorfold, Strata and oMLX, we can now run GPT 5.2-class models at 100-200 tps locally.

This is one of the reasons Nvidia has awoken and acquired labs and started releasing its own Nemotron base models. Not only hardcode enthusiasts, but soon individual laptops users will be able to run competent on-device models. The awareness of frontier labs training on customer data also makes it more attractive to purchase a GB300 or other systems to run GLM 5.3 Flash for a team of developers (or perhaps, data leakage makes cloud inference not viable for a lot of businesses).

If inference will to a greater extent happen on-premise or on-device, both hardware makers and model labs will want to win that battle, too. Nvidia is spending billions to win local, DeepSeek is rumored to release a smaller local model (30B-ish or120A10B-ish?), and it is likely more labs will follow.

The popularity of the new generation of Flash models also contributes to the growth of local. Because of the price/performance ratio of DeepSeek V4.1 Flash, my token consumption in the past month has grown to >40B and spend on DeepSeek to around $800. Being able to replace this cost is a major reason why I've ordered an M5 Ultra 256GB. Flash models replace frontier models for some workloads, and then themselves get replaced by local inference.

DeepSeek might become a platform

In the leaked recordings from DeepSeek's investor meetings this summer we learnt that the company doesn't seem overly concerned with making inference their main profit generator, even though their revenue has grown steadily over the past 2 months and their inference margins are supposedly around 80%.

If their goal doesn't stop at just selling tokens, then what is the goal? The release of the CUDA-compatible DeepGEMM kernel library, and then the Huawei-compatible DeepGEMM-Ascend might be a clue. Perhaps what DeepSeek is aiming to become is an inference engine that supports both Nvidia as well as the many Chinese GPUs and TPUs from Huawei, Moore Threads, Cambricon etc. They probably have the most expertise in cross-platform infra out of all labs.

Z.ai and Moonshot are becoming OEMs

How do labs allocate a finite number of compute hours between training new models and generating tokens from existing models to generate revenue? If you don't sell token, you might not get funding next round, but if you don't train models or if your models are not competitive, nobody will buy your tokens.

This contradiction can never be solved, and as more labs enter the market, I believe that all but one or two labs will end their inference provider businesses, and focus on just training models.

We have already seen the first hint of this, where Z and Moonshot take a 30% cut of inference revenue when a third party provider serves their models. We have already seen Devin and Harvey train their own model based on Kimi K3. Here, Moonshot looks like an OEM that offer a model to other companies, that put a coat of post-training on it and a new brand name before they sell it to their own user base.

Because labs are neither capable of marketing or sales, it naturally follows that it is better that they focus on training base models and sell those models to enterprise companies that can post-train on their own data and pay a license to host the model on-premise, and distributors like Fireworks and OpenCode who market and sell to end-users.

The same day I started writing this article we received two new open base models from Nvidia and Meta open sourced Muse Spark. As I said in April, open source models has nothing to do with China specifically, it is simply a strategy for challenger labs to gain traction in a highly competitive market. The reason China open sourced first was due to being in the position to build models in the first place, thanks to having access to the capital for training models that is not available in Europe, and even challenger labs in the US have not had the resources to train frontier models. 

As we continue on this trajectory and even accelerate competition we will see an avalanche of open source models over the next six months, and a majority of them will probably come from the US and a few from Europe. So, we'll no longer talk about open weight as something specific to Chinese labs.

Local AI is now real and the first point of attack. We're likely to see lots of competition in the 30B to 120B models that can run well on Macs and Strix Halos and RTX and DGXes.

The goal is distribution for future commercial products, and for setting standards for new architectures. I said in April that we might see new models released with paired memory, and the Engrams from Qwen and DeepSeek may turn out to be just that. Owning memory and packaging it with the model creates incredible moats, and if you can let a cloud model access the same memory as the local model then that is a massive advantage for a lab.

We'll see a ton tool-like models like Jev for all sorts of tasks, to the point where we'll feel burdened by the choice of which ones to use. 

To deal with the avalanche of models we will probably do two things:
- Abstract away the model and inference runner, and install models as packaged runtimes, and let agents access them as tools
- Build some sort of adapter that modifies the behavior of a base model; a 9B model that can be turned into a decision model or other types of models by putting an adapter in front of a request is a likely antidote to having 20 specialized models lying around on disk.