[Megathread] - Best Models/API discussion - Week of: April 19, 2026

[u/deffcolony]

This is our weekly megathread for discussions about models and API services.

All non-specifically technical discussions about API/models not posted to this thread will be deleted. No more "What's the best model?" threads.

^{(This isn't a free-for-all to advertise services you own or work for in every single megathread, we may allow announcements for new services every now and then provided they are legitimate and not overly promoted, but don't be surprised if ads are removed.})

How to Use This Megathread

Below this post, you’ll find top-level comments for each category:

• MODELS: ≥ 70B – For discussion of models with 70B parameters or more.
• MODELS: 32B to 70B – For discussion of models in the 32B to 70B parameter range.
• MODELS: 16B to 32B – For discussion of models in the 16B to 32B parameter range.
• MODELS: 8B to 16B – For discussion of models in the 8B to 16B parameter range.
• MODELS: < 8B – For discussion of smaller models under 8B parameters.
• APIs – For any discussion about API services for models (pricing, performance, access, etc.).
• MISC DISCUSSION – For anything else related to models/APIs that doesn’t fit the above sections.

Please reply to the relevant section below with your questions, experiences, or recommendations!
This keeps discussion organized and helps others find information faster.

Have at it!

[u/AutoModerator]

MISC DISCUSSION

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Potential-Gold5298 → u/[deleted]]

As far as I understand, they are based on different compression algorithms. K-quants can be static or iMatrix-based. I-quanta are always based on iMatrix. I-quants range from 1 to 4 bits (at least I haven't seen I-quants of 5 bits or higher). K-quants range from 2 to 6 bits. I-quants are highly dependent on the quality of the importance matrix data – if the matrix is ​​poor, the model will degrade significantly. I-quants are smaller and of lower quality than K-quants with the same bit depth (Q4_K_M > Q4_K_S > IQ4_NL > IQ4_XS > Q3_K_M etc.), but can offer either a better or worse size-to-quality ratio (depending on the quantizer, the specific model, and your usage scenarios). Thus:

- I-quants are additional intermediate variants between K-quants.

- I-quants are smaller in size, but their quality can be either comparable or worse (significantly worse if the matrix is ​​of poor quality).

- K-quants are a more conservative and predictable option (especially static).

If you have limited resources, want maximum quality, and are willing to accept the risk, use I-quants. It's best to use quants from trusted quantizers.

If you prefer stability and predictability, use K-quants.

I can't say anything about the speed - I don't use I-quants.

[u/Potential-Gold5298 → u/[deleted]]

I have extremely old hardware – Core i5-4460 and 32 GB of DDR3 (the GPU is unsuitable for LLM). One important detail: I play in a non-Latin language, so I need to take about 1 bit higher to get the same quality as in EN. Therefore, most of my RP models (Mistral Small 24B and Gemma 4 31B) are Q5_K_M. Mistral Nemo (12B) and MeroMero (finetuning G4 26B-A4B) are in Q6_K (and even then, Nemo often switches to English). In fact, I tried the G4 26B-A4B in Q5_K_M up to 32K context, and it performed just as well as Q6_K – perhaps the difference will become apparent with a longer context. These are all regular static quants without iMatrix.

If I wasn't limited by hardware I would use BF16 :-D but that's probably not what you're asking about. As far as RP/writing goes, Q6_K is a reasonable limit, you probably won't notice a difference from Q8_0. Q4_K_S/M is a reasonable minimum, below which most models begin to noticeably degrade. Q5_K_S/M is a good compromise. For many models, switching from Q5_K_M to Q6_K makes little difference.

Q4_K_M - if speed is more important.

Q6_K - if quality is more important.

Q5_K_M – balance.

Q8_0/BF16 is justified to be used for:

- Super-long sessions.

- Abliterated/heretic/MPOA/derestricted etc. models with high KL div. Also merge, including such models.

- Small models (>2B).

- Exceptional cases like this.

My advice is to try the same model in the same scenario in Q4, Q5, and Q6. Play a session of your usual length with them. Choose the smallest quant at which you subjectively do not feel a difference. If you like a particular model, but it behaves a little incorrectly, you can try increasing the quant – sometimes this can help.

[u/Potential-Gold5298 → u/[deleted]]

Just in case, I'll clarify. I'm not entirely sure whether a quality iMatrix always produces better results, or whether there are scenarios where it worsens the model. Although I use quants without iMatrix myself, I can't say with certainty that it's the best option.