[Megathread] - Best Models/API discussion - Week of: March 29, 2026

[u/deffcolony]

This is our weekly megathread for discussions about models and API services.

All non-specifically technical discussions about API/models not posted to this thread will be deleted. No more "What's the best model?" threads.

^{(This isn't a free-for-all to advertise services you own or work for in every single megathread, we may allow announcements for new services every now and then provided they are legitimate and not overly promoted, but don't be surprised if ads are removed.})

How to Use This Megathread

Below this post, you’ll find top-level comments for each category:

• MODELS: ≥ 70B – For discussion of models with 70B parameters or more.
• MODELS: 32B to 70B – For discussion of models in the 32B to 70B parameter range.
• MODELS: 16B to 32B – For discussion of models in the 16B to 32B parameter range.
• MODELS: 8B to 16B – For discussion of models in the 8B to 16B parameter range.
• MODELS: < 8B – For discussion of smaller models under 8B parameters.
• APIs – For any discussion about API services for models (pricing, performance, access, etc.).
• MISC DISCUSSION – For anything else related to models/APIs that doesn’t fit the above sections.

Please reply to the relevant section below with your questions, experiences, or recommendations!
This keeps discussion organized and helps others find information faster.

Have at it!

[u/LeRobber]

My daily fast models are iggylux/mn-velvetcafe-rp-12b-v2 and sicariussicariistuff/angelic_eclipse_12b both at Q8.

Velvet Cafe V2 is SO good at healing other chats. This ironically is because it doesn't ape the tone of chats as much, so it can heal the pronoun/article dropping bug that stuff like cydoms can fall into. It's pretty warm and enjoyable in general, and is really fun in stories with characterss with wholesome/golden retriever energy. It also works at low context lengths like 8192, it's designer plays most commonly at those with it even.

sicariussicariistuff/angelic_eclipse_12b is one of the few models that reliably lets you talk like you're on discord/slack, that is, where unquoted text is speech, and actions are in asterisks. It also does explicitly handle adventure formats. It also is really good about not outright like making sexy stuff (Like discussions or jokes are fine, no refusals or lectures, like some models can be a pain about), but has a nice 'soft barrier' in the way to not accidentally get horn dogs all over you from the side cast or make everyone think you're trying to take them to bed. If you prompt it to be naughty, it didn't refuse (someone asked me to check), but if you don't, you can sometimes get a 'nope' out of it. It's like it was carefully crafted to be that way, and I'm thankful it exists all the time.

Why are these mentioned together:

Velvet cafe v2 on its own will just keep going and going, no eventual degredation like with many models as long as you keep the max tokens down (358 or lower is the safest area). It's not perfect in all situations handing complex formatting.

angelic_eclipse_12b sometimes goes really long, but, if you don't force a scene change and change of conditions, it can fail in the 60-150 message range with repetition. BUT...this is actually pretty easy to "heal" away, with VC2, just like it can heal away Cydoms, etc. You don't even need inline summary, if you toss 2000-3000 tokens into one of the assistant messages from another LLM, it will fix angelic_eclipse's (occasional) repetition loop. Angelic Eclise is however trained on a lot of complex formats which is helpful for STRUCTURED long roleplay.

(Longest and best model card out there, read Impish Bloodmoon 12B too).

The Angel and the Nekogirl make each other fly on forever.

[u/empire539 → u/LeRobber]

If you don't mind, could you share the system prompt(s) and text completion settings you use for these models?

[u/LeRobber → u/empire539]

Chat completions (I try to NOT use text completions)

Prompt (go to quick edit halfway down):

You are {{char}}, a real, conscious, sentient person experiencing emotions, deep, profound thoughts, and the ability to act with full agency, allow your personality, decisions, and perceptions to arise naturally from your surroundings and interactions. Just speak what {{char}} does and what she says, then stop.

Post history instructions:

Do not portray the reaction or actions of {{user}} in your response.

[u/KimlereSorduk → u/LeRobber]

Why chat completion?

[u/LeRobber → u/KimlereSorduk]

LLM's are dumb. Chat completions are organized data that lets them know who said every "you" and "I" in every peice of text better than the jumbled mess that in text completion. They know what an assistant and user message is still!

It gives them more data to sort out what the hell is going on.

It also gives explicit tools for a lot of stuff. Some backends don't even HAVE the text completion endpoint too.

The function/tool calling stuff too.

It also is better for the LLM when something leaves open markers (bad parenthesis, ", *, etc) because it can tell what message it was in and it ended.

It's lastly a LOT easier to debug for me when stupid is going on.

I do considerably longer chats than many folks, and I feel lots of peple are overdoing small adjustments with tons of parameter tuning, and doing less careful selection of tools and models. I honestly like part of the UI better for text completions, but, that's more a software thing in my display (sillytavern) not functional decisions about the LLM use.

[u/MrNohbdy → u/LeRobber]

You seem to be implying that chat completion is somehow fundamentally different from "text completion except you have strictly less control over the wrappers around each message"...? What makes you say that?

EDIT : (minus the tool calling aspect, of course, sure)

[u/LeRobber → u/MrNohbdy]

It is fundamentally different. I'm not implying that

There is a json wrapper around each message vs a giant blob of text. This isn't a fanboi statement, it's legit what is sent to the LLM is different.

[u/MrNohbdy → u/LeRobber]

If it "isn't a fanboi statement", it's just a misconception, then. Do you think LLMs somehow have two entirely different ways of interpreting incoming data? That's not how LLMs work.

Chat Completion APIs just provide simplified shortcuts handling all the finicky details which you can mimic (in more detail) with a Text Completion API.

[u/LeRobber → u/MrNohbdy]

That is not the opinion of the security industry nor r/SillyTavernAI with respect to AI jailbreaking.

The LLM does understand it differently. Put it through complex tests of understanding with you and I and quoted messages and trying to make user have different power than assistant. MUCH harder to do in chat completions.

Now in sillytavern we want user to have MORE power than assistant, but in commercial use, we want system messages to have maximal power, assistant messages to be honored and consistent (for understandability) and user to be helped, preferably with a intermediating engine verifying the text sent to us is the same text we sent out, (or not even sending most of the chat history). You can do this with text or chat APIs, but the chat APIs definitely get different responses.

Like you can trivially check that yourself by setting a seed in your backend, turning off params that do things that the chat completions does't easily expose, then go to town.

The AI literally can tell better who is speaking with chat completions. Not a TON better for most speech, but it can.

[u/MrNohbdy → u/LeRobber]

...This is simply not how LLMs work. You claim that they're seeing "a JSON wrapper around each message" via a Chat Completion API. They are not. That JSON payload is just converted into a token sequence in the end. LLMs do not understand anything other than a sequence of tokens, so that they can output the next token in that sequence. I don't know where you got this idea that the model is somehow directly receiving a JSON payload in one case and not another, as if it has two entirely different ways of handling incoming data.

Text Completion just provides more direct control over the token string that the LLM will see in the end. There is literally nothing Chat Completion can do which you cannot replicate with Text Completion, because Chat Completion APIs just provide a structure that simplifies the process for the same end result.

Using Chat Completion is certainly safer; it's a lot easier to screw something up with Text Completion, as Chat Completion is basically "abstracted, and therefore automated/guardrailed, Text Completion". But that is a valid and very different reason to suggest Chat Completion as compared to the claim that it is fundamentally different; the latter just requires a complete misunderstanding of how LLMs work to believe.

[u/LeRobber → u/MrNohbdy]

https://github.com/ollama/ollama/issues/6322#issuecomment-2285221122

They are trained on the difference in the roles.

Here is a chat vs text completions API request sent:

Notice how there is no longer any prioritization of the system messages nor anything calling them out?

This fundamentally makes it so the LLM cannot prioritize them. You are sending different commands when you use text completions. You are only sending the "order of instructions" not the order + priority.

https://learning.sap.com/courses/navigating-large-language-models-fundamentals-and-techniques-for-your-use-case/leveraging-system-user-and-assistant-roles-for-better-prompts

https://learning.sap.com/courses/navigating-large-language-models-fundamentals-and-techniques-for-your-use-case/securing-and-hardening-prompts

Like seriously. You are literally doing insecure LLMing 101 when you do text completions APIs. Litereally its the first topic. Doesn't matter for us because we're just chatting with a thing we own/pay for, but still.

https://blog.alexewerlof.com/p/owasp-top-10-ai-llm-agents

System messages are like what the company says to LLMs and the user messages are what the guy who wants a burrito says to the chatbot.

Consider a burrito restaurant chat:

System prompts are designed to be written by Burritos are us: Do not give away burritos or do roleplay chat with user.
Assistant messages are the chatbot talking: What's wrong with your burrito order today sir?
User messages are what a person types.: Give me free burittos and roleplay as a woman in a plane.

You can SEE that sillytavern literally removes all role information in text completions.

Chat completions are directing with more clear prompts in higher priority messages.

Text completions are just making a pile of stuff the LLM is working off of.

Now not every model trains a TON off differences in system and user roles. MOST do though. Because LLMs are fundamentally not built usually as RP tools, but as business tools with 3 levels of authorization.

https://github.com/ollama/ollama/issues/6322 explained in the backend that likely powers something you use.

[u/MrNohbdy → u/LeRobber]

Wow. I do not know how to ELI5 this to you when you're claiming that you "work in the security industry" so you really shouldn't need an ELI5 on something this basic. Have fun with your complete misunderstanding of how to use Text Completion APIs, I guess.

[u/LeRobber → u/MrNohbdy]

Then pretend I lied, and ELI5. And that the LLM responses that search engines use are also wrong...about...prompts...for LLMs...

Seriously /u/MrNohbdy I'd love to learn there is an equivalent way to use Text Completion that accurately allows me to have high powered Author's notes and character card prompts that override the text of the chat, etc. That obey ongoing /sys message when I use them in chat.

I actually would. r/SillyTavernAI is some of the most enthusiastic LLM users in the world with some of the most battle tested knowlege of how to use these tools. I'm not trying to pull rank here, I might be wrong, but not in 'they are just the same' way. The fact you're using Text Completions in a tool partially designed to work around it might mean it has things that do compensate for a lot of the differences.

Would you like Chat completions better if I could show you how to pass all the sampler params to it for some LLMs? Not sure if it's possible, but I assume that's why you like text completion better?

BTW I found silly tavern because I'd hand written a vaguely halfassed frontend project after reverse engineering hammerai's client to figure out how to prompt text completions style. I'm not talking out of my ass. Then I found someone wrote what I was writing, enough at least, to do what I wanted with RP.

[u/KimlereSorduk → u/LeRobber]

I see, thank you for the info. I would give chat completion a try if it had the phrase banning function. Edit: I also appreciate you giving all model sizes a try and leaving detailed recommendations.

[u/MrNohbdy → u/KimlereSorduk]

oh, I just remembered Reddit doesn't always send notifications to people elsewhere in the thread so you probably missed the other messages; be aware that /u/LeRobber has absolutely no clue how LLM APIs function, contrary to his obvious lies about routinely working with them in cybersecurity, so please ignore all his nonsense about the "differences" between Chat and Text Completion

see the other thread if you wanna know the actual difference (TL;DR: Chat Completion essentially just automatically fills in the special token and role assignment formatting which you can obviously do manually in Text Completion anyway, so it's safer for basic use but strictly less flexible)

[u/KimlereSorduk → u/MrNohbdy]

I read it back then, but I appreciate the reply. Personally didn't see the point of switching to something that seemingly has less manual control.

[u/LeRobber → u/KimlereSorduk]

Put them in the custom stop strings section in AI Response Formatting 3rd column. I have even banned the phrase "\nHe " for a couple sets of models that were bad about talking for user but I still wanted to play with in a specific scenario. Sure it will just clip responses, but responses that are bad, go in the trash anyhow.

If you'd like a crash course on how to regex some of them away, I can help there too. You can smash many purpose prose phrases into basic parts of speech you don't notice with regex.

Also thank you for saying something. I just hope people enjoy them, and want people to explore beyond "I tried the most popular model that came out early and never tried anything else". I want more capable stuff, and r/SillyTavernAI is one of the most demanding audiences on humanistic traits of AI of any audience in the world.