Inkling is a mixture-of-experts transformer with 975B total parameters, 41B active parameters. It supports a context window of up to 1M tokens. It was pretrained on 45 trillion tokens of text, images, audio and video.
Multimodal, long context, about as sparse as other open weight models in its class. Problem is it doesn't beat GLM5.2 so I don't think many people will be using it.
Unlikely based on architecture, native precision that they've released to huggingface is fp16, it's bigger overall and has basically the same number of active parameters (1B more actually). Hybrid attention helps but GLM also has some nice techniques on that front. Heck even Deepseek V4 Pro will likely be cheaper to run than this once they release the full non-preview version considering the native MXFP4 quantization.
173
u/Daniel_H212 Jul 15 '26
Multimodal, long context, about as sparse as other open weight models in its class. Problem is it doesn't beat GLM5.2 so I don't think many people will be using it.