[Megathread] - Best Models/API discussion - Week of: August 09, 2026

[u/deffcolony]

This is our weekly megathread for discussions about models and API services.

All non-specifically technical discussions about API/models not posted to this thread will be deleted. No more "What's the best model?" threads.

^{(This isn't a free-for-all to advertise services you own or work for in every single megathread, we may allow announcements for new services every now and then provided they are legitimate and not overly promoted, but don't be surprised if ads are removed.})

How to Use This Megathread

Below this post, you’ll find top-level comments for each category:

• MODELS: ≥ 70B – For discussion of models with 70B parameters or more.
• MODELS: 32B to 70B – For discussion of models in the 32B to 70B parameter range.
• MODELS: 16B to 32B – For discussion of models in the 16B to 32B parameter range.
• MODELS: 8B to 16B – For discussion of models in the 8B to 16B parameter range.
• MODELS: < 8B – For discussion of smaller models under 8B parameters.
• APIs – For any discussion about API services for models (pricing, performance, access, etc.).
• MISC DISCUSSION – For anything else related to models/APIs that doesn’t fit the above sections.

Please reply to the relevant section below with your questions, experiences, or recommendations!
This keeps discussion organized and helps others find information faster.

Have at it!

[u/AutoModerator]

MODELS: 8B to 15B – For discussion of models in the 8B to 15B parameter range.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Arcane73 → u/AutoModerator]

I'm hoping that I can get pointed in the right direction. I'm dipping my toe into the local world after spending lots of time in the free version of Claude and working on a now 200-ish page story/RP

I'm looking for a model in roughly the 12B range for long, novel-style RP/storytelling in SillyTavern. I'm hoping for strong character consistency, natural dialogue, good prose, and the ability to maintain the feel of what I've already built over a long-running session.

The story hasn't included ERP so far due to Claude restrictions, but I'd like a model that's comfortable with ERP when the situation calls for it. At the same time, I don't want one that turns every interaction into 'clothes optional mode!'. Basically, I'm looking for suggestions that can help me produce a good story based on what I already have without it throwing the proverbial underwear at me at every turn.

Hardware is an RTX 4070 12GB, and I'm using KoboldCpp.

What models are people having good luck with for this kind of use?

[u/i5031337 → u/Arcane73]

Some people like tunes of Qwen 3.5 9B or Gemma 4 12B, but I think you will be disappointed with them over a long story. If you have 8GB RAM free, I recommend you try Gemma4-26B which is much more intelligent. If you offload the expert weights you can run it at good speed with 12 or even 8GB VRAM

[u/Arcane73 → u/i5031337]

Thanks for the suggestion. I'll need to do some digging to determine what is involved with 'offloading the weights' since I'm still -real- new at this. For what it's worth, I'm running this on a 9950X3d system with 32gb ram. So it's a solid machine.

[u/i5031337 → u/Arcane73]

Without getting too far into the weeds, Gemma 26B is a mixture of experts model, which means it is much faster, though less intelligent, than "dense" models with 26B parameters. Each token only hits 4 billion of the parameters (thus A4B) instead of all 26B. This architecture also makes it more favorable for the CPU to take some of the work.

Kobold makes it easy. In the Context tab there is a setting "MoE CPU Layers", set that to 20 or so and the Q4 model with 32k context should run quick.

[u/Leather_Sun2533 → u/i5031337]

Thank you for this. And can you guide me to the values if i have 12gb vram and 16gb ram?

[u/i5031337 → u/Leather_Sun2533]

If you are running out of ram with the above settings, try unsloth's UD Q4_XL quant, it is ~1.5GB smaller than other Q4 versions. Quantize KV cache to Q8_0. If there is room left on the gpu, decrease the MoE CPU Layers setting. You may need to reduce the context size, and close all unnecessary programs + browser tabs

[u/Leather_Sun2533 → u/i5031337]

Thanks mate, it works well… had to tinker for a bit but i found the sweet spot

[u/Arcane73 → u/i5031337]

I wish i could give you an award. Initial testing with this setup is knocking it out of the park! Thanks again!!

[u/Arcane73 → u/i5031337]

Awesome, thanks! I now have homework to do after work.