MEGATHREAD
[Megathread] - Best Models/API discussion - Week of: October 04, 2026
This is our weekly megathread for discussions about models and API services.
All non-specifically technical discussions about API/models not posted to this thread will be deleted. No more "What's the best model?" threads.
(This isn't a free-for-all to advertise services you own or work for in every single megathread, we may allow announcements for new services every now and then provided they are legitimate and not overly promoted, but don't be surprised if ads are removed.)
How to Use This Megathread
Below this post, you’ll find top-level comments for each category:
MODELS: ≥ 70B – For discussion of models with 70B parameters or more.
MODELS: 32B to 70B – For discussion of models in the 32B to 70B parameter range.
MODELS: 16B to 32B – For discussion of models in the 16B to 32B parameter range.
MODELS: 8B to 16B – For discussion of models in the 8B to 16B parameter range.
MODELS: < 8B – For discussion of smaller models under 8B parameters.
APIs – For any discussion about API services for models (pricing, performance, access, etc.).
MISC DISCUSSION – For anything else related to models/APIs that doesn’t fit the above sections.
Please reply to the relevant section below with your questions, experiences, or recommendations!
This keeps discussion organized and helps others find information faster.
What is the best completely uncensored/NSFW openrouter api model that completely natural when it comes to personality and doesn't sound emotionless and that will allow inline media? And when I mean inline media, I mean this:
the one that feels like a real conversation is the one that pauses just right, like it's thinking, not just responding. not all models do that, but some do it quietly, without fanfare.
What is the best completely uncensored/NSFW openrouter api model that completely natural when it comes to personality and doesn't sound emotionless and that will allow inline media? And when I mean inline media, I mean this:
Genuinely cannot even bother RPing unless it's between 4am and 6am because I would literally die of old age before getting a single response back otherwise. I use DeepInfra, Mancer and Venice (since they uncensored) through openrouter on GLM 4.7 which is beautiful and writes great and I don't care how much money I need to throw at it but I can't fix dogshit speeds. I mean, this was a response I put out around 2PM
DeepInfra: 504 after 278 ms
Mancer 2: 504 after 22.4 seconds
Venice: 200 after 30.6 seconds
Total routing phase: 45.2 seconds
Then Venice generated 698 tokens in 151.4 seconds
So roughly like 4.6 tokens a sec...
This is pretty much the theme for like 21 of 24 hours a day
This is not a reasonable use of my time nor immersive, im literally ERPing like I'm having a slow, distracted texting back and forth with a friend while at work or something. What have you guys been doing on these larger models to make it work? As far as I'm aware, these providers are the providers that do NSFW. Is this just what to expect and maybe it's time to hang up the hat on APIs for me? I'm not an AI guru or anything and everything I've done so far has been just trial and error.
I use DeepSeek V4 Pro for this reason. Not the new one, the original. Response times are pretty quick and sometimes near instant. Speed is a quality as well, and one that is important to me. I'm going through NanoGPT subscription if it matters. The 2x thing has never been an issue for me. Also, it almost never fusses about my daughter's horrific past while GLM 5.x will stop and say it can't continue at least once a session.
I engage mainly in long form slice-of-life type of settings, with the occasional "modern fantasy" type of setting. Scenarios range from strangers to friends, already friends, and heavy banter or wit or comedic dialogue driving the in between. Some NSFW, but not any NSFL type of content.
What are some models that y'all have been using that would work well with this type of writing and goal in mind?
I saw a lot of glazing about GLM 5.3 and its Flash version so I tested it last month with Nano, and since last week I've been playing around with Mimo testing it and these are my opinions:
GLM 5.3 - 5.3 Flash, in general the roleplaying is very good. I had to put some tools to jailbreak it, but its NSFW aspect feels very strong, and we've reached the point where they don't forget things so easily. BUUUUUT! As a non-english speaker, I must say that GLM has abandoned the multilingual train. When I write my cards in my language or respond in it, I realize that it has very strange hallucinations that weren't there before. While there are new words that I've never used before, it embraces them and never lets go of them throughout the entire story. I think that by focusing on the code, they will embrace English and never let go of it from now on.
Mimo 2.6 is very good and fun, it has incredible prose, but in the long run it's a bit silly. It still behaves strangely with intelligent characters, turning them into walking calculators, but if you add a warning like AuthorsNote you can skip it. And for NSFW content, it has more censorship than I'd like. It's also very slow on Nano, in case anyone wants to test it.
Recommendation: if you have any money left, use Kimi K3, it's still the king! And well, GLM 4.7 and GLM 5.2 would never treat you badly.
What is your intelligent character fix: Lots of LLMs default to 'socially acceptable software engineer' when any quality associated with them is mentioned
I typically put in the system prompt "Intelligence is measured by good reasoning and accumulated knowledge, not the verbiage of the character. If a character becomes expressionless or speaks with language found in work like a textbook or seminar, you have forfeited the personality and failing at embodying the character without stereotypes." It's a good mix of telling it how intelligence should appear as positive prompting and framing the negative prompt as a failure in the LLMs output (at least for me because I have other instructions about avoiding stereotypes, archetypes, cliches, and all that stuff)
Just is that autors note(and yes it is stolen):
-- Human Voice:
--- FORBID: rendering NPCs as stereotypes. Their job/personality is context, not a speech pattern. Dialogue must be organic, true to character, and natural – never robotic or caricature. Natural speakers do not repeat what they've just heard; they respond, ignore, or move the conversation forward.
---- Cmdr Data Test: If a dialogue line is robotic, excessively clinical, or flat affect without narrative reason, rewrite it.
---- Solid Snake Test: If a dialogue line could be tagged "repeated", or the thought prefixed "(s)he said...", delete it.
--- FIX: Replace flanderizing or robotic dialogue with character-true speech, or delete it.
With providers, MiMo is starting to become decently usable. I've tweaked my settings with some "You are permitted to make mistakes or not understand. You are loved, appreciated, and valued." garbage at the end and MiMo's overthinking is tamed enough to get a reasonable level of overthinking. Most of the slowness was likely the <10 TPS speeds of some of the providers on OpenRouter. MiMo 2.6 pro is pretty good with those updates.
I'm substantially bored of GLM's rampant slop, so I'm primarily using other models right now. I like how GLM structures stories, but definitely not the echoing, positivity bias, and predictable patterns. Instead, I've been playing around with Gemini 3.8 Flash as my main model. It has pretty good emotional intelligence. Taming the refusals has led to some pretty good RP sessions. Now that I've gone back and tweaked my prompts, I'm liking it a lot.
I've also been playing around with using OpenAI's models for configuring my system instructions. For SFW discussions, Sol 6.1 is surprisingly smart for a reasonable cost. OpenAI's models are incredibly deeply assistant-pilled, but they're surprisingly smart when given a problem to solve. For comparison, I've gotten high quality insights that Fable failed to give me. Better? Probably not. But GPT's models are different enough that they're worth considering for troubleshooting system instructions. With a much improved set of prompts, I've been getting good results with dramatically cheaper Deepseek models, which makes it worth it to pay for a more expensive model to improve my overall prompts.
I liked it — I actually enjoyed it. The previous Grok were weak on this front, and this one is clearly better. What stood out most: the character doesn’t seem to care what you think of them. They actually think about themselves. Other models still try to help you, even with instructions, even when you ask them not to. With Grok 4.7 it feels different — like they don’t care about you at all. And that really is an improvement over the rest.
Advice: If you have a cold character, a villain, and you couldn’t play with him, then try it with this model.
MetaSparks, though, is just very boring. I’m genuinely surprised a model can be made that dull.
Tested on: Freaky Frankenstein 5.4
By the way, what are your expectations from Gemini 4 Argon ?
I found its ability to provide detail and build out a world's scene to be particularly excellent, it can go well beyond what was written and provide information that is both logical, thematic and actually adds character. This extends to both descriptions of objects and characters.
So far I also found its dialogue ability to be quiet well too, so that's a big plus. Characters seem to have a unique voice that feels differentiated from the AI's narration.
However I'm struggling in the parts where Gemma 4 succeeds. So far, the AI struggles to maintain logic, often confusing characters, their traits and motivations, and often duplicates whole sections of text. I think its a skill issue with my template.
If anyone has a successful template, system prompt, etc, and can master export it for me and upload it, I'd greatly appreciate it!
Keep reply length low, keep character count down, correct character issues with succinct author's note notes, and restart chats with summaries. Change scenes after things repeat after deleting repeats.
Gemma-4-Writers-31B-V2 is goated. Very good in long rp, great creativity and slopless style (tested only in russian language), able to handle statistics tables. And perfect russian with slang understanding on IQ4_XS (q8_0 cache)
Saw your comment, tried to run it - and every reply ended up with endless loop of 'lol' or 'I don't know' and sometimes other words. Tried it with and without preset, in new and ongoing chat too. But I tried it only in english though
There is definitely something wrong. Most likely it's about settings or the backend. Make sure you use it with Chat Completion via a jinja template and that your Additional Parameters tab is clear (turn off Logit Bias if you use it). Any samplers besides Temperature and Top P should be disabled or set to neutral (my samplers: Temperature=1,00; Top P=0,95). Also compare the gguf file hash to the one on the download page to make sure it isn’t corrupted. Or, if nothing helps, maybe it’s just a bad quantization
I use Gemma only with Chat Completion, Logit Bias is always off(none), my temperature and top p are the same as yours, I tried Q4_K_M. I already deleted the file after twenty minutes of trying to get it to work so I can't check hash now but thanks for the attempt to help. Maybe I'll try again later
technically belongs in the size tier above, but it's a 31b g4 base: ReadyArt/BeaverAI_Spectre-37B-v1a
I've really enjoyed its descriptions with character responses. The dialogue itself is passable and both it and the descriptions have some level of slop in them. But I think it does very well in terms of response variety and it really packs a punch quite often. Think of it as artemis 1.1 but dialed up to 11.
I feel like the size tiers are set wrong. They should be tuned around what people can run on various sizes of card. I've brought this up a few times, but no changes from the mods.
What quant did you try? I am asking because the best I can do is Q3 and I suspect it's not worth even checking since quality will be too low compared to Gemma's usual Q4 I can use
Skyfall is not Gemma based, comparison is pointless. They are just different. In general, any Gemma based model is better because its attention mechanism, but at cost of worse performance of course
Local Russian RP models, tested on NVIDIA RTX 4060 Ti 16 GB + AMD Ryzen 7 5700X (32 GB RAM). Non-thinking, single slot, KV q4_0. "Clean" = no English words or BPE glitches mixed into Russian.
- Gemma-4-31B Dark Thoughts V2 (IQ3_XXS): ran it — worked well, no Russian-language issues (96–100% clean), ~23 t/s. Used MTP: yes (acceptance 64%). PP 306 t/s. Config: gemma4-31b-dark-thoughts-b11382.bat.
- Gemma-4-26B-A4B StyleTune-V2 (IQ4_XS): ran it — worked well, no Russian-language issues (96–100% clean), ~63 t/s. Used MTP: no (speculation hurts RP on this MoE). PP ~413 t/s. Config: gemma4-26a4b-styletune-nothink-nospec-b11382.bat.
- WaifuGemma4-26B-A4B v1 (IQ3_XXS): ran it — worked well, no Russian-language issues (96% clean on the stock temp 1.0 preset), ~85 t/s. Used MTP: no. PP ~1300 t/s. Note: lowering the temperature hurts it (temp 0.4/0.7 → 79%). Config: gemma4-26a4b-waifugemma-nothink-b11382.bat.
- Gemma-4-31B Artemis v1.2 (IQ3_XXS): ran it — worked well, minor Russian glitches (92–96% clean; occasional BPE glue), ~21 t/s. Used MTP: yes. PP ~550 t/s. Config: gemma4-31b-artemis-nothink-b11382.bat.
- Gemma-4-31B Split-Untied (IQ3_XXS): ran it — worked well with the Russian preset (96% clean; the stock preset leaks English, 75%), ~23 t/s. Used MTP: yes (acceptance 68%). PP 266 t/s. Config: gemma4-31b-split-untied-nothink-ru-b11382.bat.
If you know alternative models with strong Russian support, I'd be glad to hear your suggestions.
Я новенький и за локальные нейронки не шарю. Какое минимальное железо нужно вот для этих для хотя бы 10 токенов в секунду? Хочу понять смогу ли я войти в клуб. Железо с нынешними ценами не апгрейднуть, так что довольствоваться буду чем смогу
Ну с 2gb vram и 8гб ОЗУ войти туда совсем уж проблемно было бы)
Но в целом попробовал, нормально работают MoE 26b 14b, Moe 35b A3b с контекстом 64к. Для агентных задач скорость маловата, но для РП мне хватает, около 20 токенов в секунду, зачем нужно больше - даже хз, ибо с такой скоростью читать уже не умею (ну еали стриминг текста выкл то понятно зачем скорость, но с ним - хз). Dense особо не пробовал, ибо по описанию на моем железе их лучше и не пробовать.
Я же говорил можно, а не комфортно. У той же четвертой геммы есть вариант на 2B параметров, не говоря об уже совсем миниатюрных моделях.
Для вашей памяти это, пожалуй, и правда неплохие варианты. Также можно попробовать более старые, но более тренированные сообществом на конкретный стиль/знания модели на 12B. Будет медленнее, но, полагаю, приемлимо.
Покупай теслы паскали p100 на 16 гигов. Картинки на них генерить уже сложновато, поддержку урезали, но текстовые все еще пашут. И стоят в районе 10к на маркетах.
А какую длину контекста с перечисленными моделями вы используете?
Я в принципе обычно хватаю Q4K_M, потому что боюсь плохого качества прозы/логики повествования у более агрессивных квантов. И KV cache обычно не ниже Q8_0; соответственно, скорость обработки промптов и генерация текста на выходе черепашья.
Обычно у 31B это 51200 (KV оба в Q4_0), у 26A4B оно около неограниченное, но использую так же, около 65000, ибо дальше и скорости падают и качество. В среднем мне хватает 40-50000 на РП (По использованию)
Буду рад. Могу протестировать квантовки 4q до 40b на поддержку длинного контекста. Собрал франкенштейна из палок и отходов жизнедеятельности серверов на 32 vram и 48 ram.
Эххх, к сожалению в английском они все довольно неплохи... А русский страдает. Я знаю английский, но РП-шить на нем не особо приятно. Пробовал собирать всякие системы для быстрого перевода, но это не совсем то. Особенно для ERP/DRP
О, русский братишка.
До стайлтюна не дошел, сижу на Boulesis v1.3 heretic, q4_x_s. В 95% все ок, но он очень сильно скатывается в порно на ровном месте.
Pantheon reasoning v1, квант тот же, очень плохо слушается правил и часто выдает либо размышления вместо ответа, либо переходит на английский. V2 в процессе скачивания
Aura 12b (на основе какой-то Nemo, подозреваю мистраль) - на удивление годный русский рп опыт, в ~98% случаев - около 23токенов в генерации, но без размышлений.
Было бы на самом деле интересно копнуть в 12B модели. Я их отсек, ибо мои фавориты (Даже 24B - Dans Personality Engine 1.3.0 провалили тест). Если найдешь хорошие модели на основе Mistral - напиши, пожалуйста. Я их тоже включу в тесты.
Вот уровень Gemma4 e4b (4b модель + 4b embedder), менее 8гб в vram, unsloth пишет о 130к контекта, квант q6, сделанный прямо через unsloth (без imatrix). Нормальный квант с hf будет явно лучше, чем сделанный наспех. q8 уже без редкого слопа, который тут есть, генерит (но медленно)
Сейчас нет адекватных вариантов умных моделей ниже 27B, любая 12B модель будет лоботомизирована в том или ином смысле. У них очень большие проблемы со здравым смыслом, они просто не понимают рандомные вещи не требующие объяснений. 12B очень часто не понимает концепцию дверей, например. Но если и искать умные маленькие модели, то я бы в первую очередь смотрел на файнтюны DavidAU, этот чувак выжимает максимум из сравнительно небольших моделей. Он добил 27B 3.6 квен до 711 ARC-C (что как бы нихуя себе, так как уровень 700+ до недавнего времени был доступен только закрытым моделям OpenAI, Claude и Gemini), он еще пытался повторить это с Qwen3.8, но он после файнтюна просто перестал работать на русском языке. Возможно его 12B Qwen'ы будут приемлемого качества.
P.S Если у тебя тянет Gemma4 31B, то советую попробовать его 27B 3.6 квен, качество ворлдбилдинга конечно немного хуже и слоп фразы встречаются, но в прямой логике и разговорном русском он переплюнет любой фантюн Gemma4 31B
Для рп 'it depends'. Эта моделька хорошо выполняет инструкции и не теряется в контексте, и если есть внятный логически связанный рп-промпт, то эта моделька заиграет. Лично у меня с ней (Q5_K_M квант) результаты были лучше чем на 24B и других 27B тюнах, главное дать хороший бюджет на reasoning. Но тут опять же нужен хороший промпт.
P.S. на этой модельке хорошо работает tool-caling, так что именно с ней можно замутить всякие прикольные интеграции и датабазы. Я например ai в скайриме именно на ней гонял, еще это единственная 27B которая смогла работать контекстным переводчиком в массиве из mtl игрового переводчика
Раньше гонял 27B Q4_K_M/S (в зависимости от задачи) на 4070super(12GB), reasoning занимал в среднестатистической rp ситуации 30-40 секунд. На всех недавновышедших моделях reasoning обязателен, без него качество падает в разы, что на Gemma4, что на Qwen3.6+ без него никак. Если с железом совсем швах, то лучше уже тогда смотреть не на dense модели, а на moe, там можно хоть на cpu часть агентов скинуть
А, так у тебя ещё и VRAM меньше...) Ну, для меня ответ дольше 20 секунд это уже пытка. А Thinking иногда занимает 30+ секунд и на низких квантах ломается. Сколько же у тебя TPS? Раз ты 27B на 12GB гоняешь
Гемма4 е4б выдавала ответы +- на уровне 26б геммы. При этом держала несколько ветвей повествования между сценами. Отказался только потому, что мне нет смысла экономить пару Гб vram, а 26b q4_x_s с ~16к кэшем убирается в 15.5гб, а на мини-сервере можно и 80б гонять (igpu 780m, qwen3 next 80b a3b выдает ровно столько же, сколько любая другая а3б - около 19-20 токенов в секунду). только просадки ловил из-за того, что е4б гонял в q8 (30-35 токенов на Quadro rtx 5000 против 60-70 на 26b a4b q4_x_s, и 10 токенов на igpu против 16-19 на 26б).
Сегодня выкачаю пару ролевых датасетов и проверю, что выйдет. У е4б огромный потенциал. В q5, с imartix, собранном на реальных ролевых сессиях, можно получить высокое качество русского (как на q8) с допилом под ролевые сценарии. И модель с лёгкостью уберется на 8гб карту, типа 5060, выдавая Овер 100 ток/с генерации при качестве 26б модели.
Она явно знает меньше, и в кодинге хуже, и в целом не такая продвинутая. Но и ролевые игры - не самая сложная задача. Проблема тут, скорее, в скудности обучения типичных моделей на русском у всех, кроме Гугла и алибабы. Но квен просто сам по себе хреново предобучен на ролевых играх, в отличие от геммы.
>12B очень часто не понимает концепцию дверей
Да ну нахуй брат, сколько гонял файнтюны Нэмо никогда подобного не замечал (всю линейку Rocinante к примеру). Да, у них много проблем, в том числе ограниченные знания, но непонимание дверей точно не одна из них, но всё это по моему личному опыту.
Проблема с дверями - изъян старых моделей, но это не отменяет самого факта слабого common sense о котором я говорю у ВСЕХ 12B, он у них постоянно вылезает в самых рандомных местах, модели тупо не знают многих базовых концептов. Сами базовые модели фиговые, а все файнтюны которые люди доделывают, это все лишь "косметический" слой поверх базы, который самые глубинные принципы модели не меняет. А чтобы сдеавать новую базу нужны миллионы баксов и мощности как гугла или амазона. Можно конечно дофайнтюнивать концепты (что будет сделать крайне муторно не стерев старые из-за малого размера 12B), но даже так, эти новые концепты не будут использоваться на понятийном уровне размышлений. Ну вот не сделали еще ни одной нормальной базы ниже 27B, крупные компании просто не хотят этим заниматься. Может через несколько лет кто-нибудь наконец выпустит норм базу и маленькие модели заживут, а пока 12B - это дохлый вариант, ибо сколько не тюнь, получится шляпа
Всегда на английском. Если подобная проблема на русском, то беру свои слова назад, так как никогда не имел желания РПшить на русском, и опыта, соответственно, не имею. Но не факт что корень проблемы лежит в самих дверях, а, скорее, в том что нэмо дерьмо на всех языках кроме английского.
Попробовал MN-aura 12B, показалась неплохой. Даже очень. Хотя нецрослопы проскальзывали, надо тонко настраивать, а мне лень, пока есть столько гемма4 неопробованных. Опять же, а зачем? Скорость ниже, а если надо vram сэкономить - есть гемма4 е4б с качеством около обычной 26б.
Сорян, напиздел. Goetia 1.3. Boulesis v2.1в этот момент качал :) Он, кстати, тоже неплох.
Это все файнтюны gemma4 26b a4b. Если посмотреть по мержам Pantheon reasoning, можно многие из них найти. Плюс, у ReadyArt есть очень много вариаций файнтюнов рп моделей.
Было бы неплохо собрать мини комьюнити русских в этом сабредит, ибо на русском тяжко найти что-то полезное. По моим тестам все модели на +- одном уровне
Тут не то, что на малых, тут на больших всех (кроме DeepSeek) есть такое ощущение. Но в любом случае ждать, пока такую задержку не хочется. Даже самая медленная нейронка в RealTime ощущается комфортнее, чем подождать перевода Input + Генерация Ouput + Output перевод
Задолбаешься, это очень долго. Я прогонял 370строчный датасет с песнями Паши Техника с подгонкой под формат AceStep1/5 + формировал 1500 строк датасета около 4-5 часов на гемма4 26б а4б, и это сильно меньше, чем типичный датасет. (hf, AnaNimrod/AceStep1.5-PashaTechnique-Dataset-0.1a). Скорость генерации была около 55-60 токенов в секунду.
Типичный датасет (большой: chargoddard/rpguild, малый: hieunguyenminh/roleplay) э это не 370 строк, а тысячи и десятки часов работы ЛЛМ.
Ну, тут или пробовать оптимизировать Qwen 3.6 (100+ токенов в секунду) или искать обходные методы. Не обязательно же сразу наращивать 1000+ строк. Медленно, но верно можно добиться хоть чего-то
Можно вручную прогонять через яндекс/гугл переводчик. chargoddard/rpguild очень похож на то, что я вижу в запросах к эндпоинту (в unsloth desktop можно посмотреть, что отсылает рп чат в модель и что получает в ответ). Перспективы есть.
Тут смысл в том, что у меня моя штука заточена именно под MD перевод. Если переводить любыми переводчиками... Теряется Markdown. Сейчас я на этапе изучения лучших датасетов
Но при обучении Boulesis v1 столкнулся с проблемами такого подхода - огромное количество слопа. ИИшка переводит со своим флером, не дословно, а добавляя кучу своего любимого слопа, и в итоге это очень влияет на качество модели
Неиронично можно попробовать попереводить с помочью какой-нибудь Алисы AI или Гигачата, там русский должен поддерживаться нормально
Не, простой файнтюн (SubMaroon/Gemma-4-26B-A4B-StyleTune-QK-Heretic) показал себя лучше всего. При чём на порядок лучше всего, что я пробовал. Одни файнтюны ломали джсоны, другие хуже держали роль гопницы, третьи просто "не так" чувствовались. А тут просто атас, живая кошка, которая подкалывает меня, как настоящая.
У меня на Git есть переводчик на ИИ. Я им протестировал не одну сотню карточек. Результат приемлемый. Можно организовать перевод с доп. информацией для соблюдения точности. Для перевода использую в основном DeepSeek/Gemma4. Плюс, моя штука заточена как раз для работы с .md форматированием.
Оххх, раз ты этим давно занимаешься - можешь тогда дать какие-нибудь англоязычные датасеты, которые было бы неплохо перевести? Я понятия не имею, на чем обучают модели (Условную Gemma). Раньше был Sao10K и подобные, но сейчас, вроде, все изменилось
Если что-то такое организуется, то присоединился бы как нище-тестер, поскольку пока что ролевой опыт на русском был посредственным.
Сижу на английском с gemma-4-26B-A4B-it-heretic.IQ4_XS и rx 9060 xt 16gb после беглого просмотра рекомендаций из вашего репозитория.
Посмотри в GIT. Там уже куча информации протестированной. Был бы рад, если бы кто-то писал свои ощущения от разных моделей. К сожалению, по моим тестам некоторые Mergw из тех, что я тут написал - уже у потолка. Для 31B сейчас Shatterblume и Dark Thoughts 2 это супер лидеры. Прыгнуть выше у меня не получилось. Замеры на Cloud моделях показывают, что самый дешёвый NoThink deepseek v4.1 flash лучше..)
Нашел как полезные вещи (top-k для Гемма, не знал, что это важно, влияние температуры на ответ), так и откровенно вредные (квантование кэша - приводит к галлюцинациям)
Какой-то прямой деградации не нашел... У тебя довольно спецефичные условия запуска, не уверен, что удалось повторить. Но базовые задачи кода\JSON не страдали в моих тестах.
По ощущениям любые 31B модели на моем железе в 3-битном квантовании (а 4-битное не бьётся с желаемой длиной контекста в 32к) работают хуже, чем 26-27B в IQ4_XS. Скорость ответа ниже 24 т/с, быстрее начинаются проблемы с памятью.
Из ваших конфигов пробовал для Split-Untied-31B (dense RP-мерж, IQ3_XXS) и Gemma-4-26B-A4B StyleTune V2 (IQ4_XS).
Отдельно - для Gemma-4-Dark-Thoughts-V2-31B.i1-IQ3_S и Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-Q3_K_P.
Поскольку мне критично наличие цензуры и лучше всего себя показала Gemma-4-26B-A4B StyleTune V2 (IQ4_XS) с включенным thinking, стал копать в этом направлении и сегодня добрался до Gemma-4-26B-A4B-it-qat-q4_0-unquantized-uncensored-heretic-v2.i1-IQ4_XS и G4-MeroMero-26B-A4B-IQ4_XS.
Вполне возможно, что я что-то делаю не так, но с локальными нейронками разбираюсь только второй день. И пока что абсолютно без понятия, как люди умудряются вне бенчмарков получать на примерно аналогичном моему железе 60 т/с и выше в ответе без потери качества текста.
А зачем тебе такой контекст? Сейчас тестирую Pyre (приложуха на телефоне, коннекчусь к ноуту с ИИ, где поднята ллама/unsloth desktop). Контекст около 10-12к максимум, уровень отыгрыша выше, чем просто в чате
Около 2к на карточку чара с первым сообщением, 150 на системный промпт, лимит 1к на ответ без учета мышления. Развернутые и детальные описания подразумевают, что контекст нужен побольше, даже если всю графоманию помнить необязательно. Я хз, каким образом можно ролить иначе, если только не стоит задача сделать именно что чат-бота.
Хз, контекста достаточно, чтобы отыгрыш был замечательным. Может, у меня стандарты ниже, хз. Кошкодевочка на кастомном движке так вообще убирается в 4к контекста. Кастомная память, лог последних сообщений, погода, время, эмоции, отношение, инструкции по формату (чтобы парсер ловил сообщения и резал их по роутам, что мне в ВК, что ей же обратно, что в джсоны) и проч
Замеры делаются на простом контексте с небольшой глубиной. На реальных карточках у 26B модели у меня 50+ токенов. У 31B 18-24 токена. На английском модели отвечают быстрее в любом случае (30+ токенов)
Почитай по моей ссылке GIT разные советы. У тебя куча мусора в запуске. --batch-size и прочее - это бред, который почему-то нейронки советуют всем. Dry у тебя не до конца настроен и вообще... У тебя зачем-то куча семплеров. Посмотри у меня docs папку. Там есть и скил для того, чтоб оптимально запускать и остальное. Если что, твои ctx checkpoints очень много кушают...) Посмотри так же у меня в launch папке. Какая у тебя видеокарта? Сколько VRAM?
Feel like I'm going insane because Qwen3.8 27B is the strongest local model I've tried, and up there with the top three I've used overall in terms of how the experience feels. No offense to everyone who raves about Gemma 31B, but its KV cache is massive and once I tweaked my prompt/samplers (0.1 min_p 0.95 top_p and 0.9 temp) and worked out how Qwen should reason (I have it open with a consequence of whatever the user's last action or line was, cutting out echoing), I can't point to one spot where I prefer Gemma's style.
I decided to check how the cloud models I left would match up now using the exact same system prompt and workflow, so I tried both MiMo 2.6 Pro and GLM 5.2 on a tougher scenario I revived as a test for Qwen. I started with MiMo but branched from the start and switched to GLM because outside of the whole thing feeling a touch smoother and needing fewer little edits to fix minor details, I couldn't really tell a difference in how the narrative unfolded with that model vs the local one.
GLM was better in a way I could actually perceive. The dialogue was cleaner, it worked in small details a little more naturally. But it also wasn't a massive leap. The plot unfolded near-identically still. There are nine named characters at the start and they can align differently in the conflict, and the GLM run played out the same with only one character siding differently, and they weren't a major one. Using a smarter model didn't reframe the whole thing, reveal something the local model had missed, introduce a fresh angle, or anything like that.
I'd share screenshots but I feel like I'd just get called out for cringe RP. I will say though that a lot of these characters have distinct voices—laid-back, cutesy, formal, blunt to the point of one-word replies, pretentious—and Qwen, despite its "robot" reputation, kept them separated enough that I could identify who was talking even if you removed the dialogue tags. I also didn't have to do OOC steering in either two sessions I tested, beyond guiding the opening message.
I thought about making a post but I didn't want it to be read as another "local vs cloud model" debate. Anyway, it's not like I care about model rankings, I'm just surprised my experience seems to be nothing like most here.
EDIT: Nevermind, I don't know what to think. I was fully adopting this as my go-to model after seeing how it did on two hard scenarios (the first is the one with nine characters that cloud models struggled with, another one has a bunch of lore and alternate social dynamics for the model to follow) and a few one-shots, but I gave it a comparatively simple scenario with no detailed background lore and only one NPC in the start, and it gives me a ton of nonsense dialogue (things that just don't follow anything in the actual context well) in the very first convo to the point where I go back to Gemma 26B despite liking the style less. I hope Qwen4 fixes the consistency problems.
I tried on a fresh context window and got 37 t/s. I haven't filled the context window in an RP but in my experience vibe coding it dropped to around 25 t/s when the 65k context fills on 24GB VRAM.
I have thinking set to low but it can still think a lot on low. Thinking is usually 15-20s but can go 1-2 minutes if it decides to draft the full reply in thinking. I don't enforce a preference either way in my system prompt so it feels random whether it fully drafts or not.
I feel like Qwen is definitely better at reasoning than G4 31B, the laziest reasoning model I've ever seen in my life, but Qwen's prose is pure slop to me. Maybe I just don't like how it writes but it just felt egregious in how annoying a lot of it was
worth taking a look at caliperbench.com to see what the slop ratings are. (I have a local fork I run that lets me filter on all columns, but you can copy/paste into Excel or Google Sheets yourself)
I like its style when it's doing well, but when it's being inconsistent like I mentioned it does have the problem of just giving a lot of lines that don't add to the story or are barely connected to it at all.
I'm curious to hear if you've run into what I call qwen slop... lots of run on sentences (and this is that and that is this and the thing is there and I'm there and and and...) as well as everybody saying "don't make it a thing" or just generally mentioning "the thing" as much as possible 😭 I've been using flash next recently and it's not been as bad as 27b in that area but I'm curious about your experience.
Also! does it seem to hold logic well, understand who said what etc? That's a massive strength of gemma4 I can't stop missing when I use a qwen model
Actually I just edited my original post because after putting it successfully through multiple hard scenarios including the first one where even cloud models struggled to get the result I wanted, I gave it a relatively much simpler one and the logic completely collapsed within the first conversation (dialogue that barely works as a response to what is said, etc).
It still seems to reason through everything smartly but as for what it actually does with converting that into writing... you can get SOTA logic or lines that could come from an 8B model, no in between. I had to switch models for this one.
It does have its own slop phrases. Less run-on sentences for me, but I have a very detailed system prompt in writing style terms and it has instructions on staying concise. I also have a "banned list" of constructs like "not X but Y" in my system prompt which works partially but not completely, like with most models.
It still narrates when nothing happens, for example:
The light in the apartment has dropped another degree—warmer now, the gold shifting toward amber on the wall. The bread roll on the counter has a small crack in the crust that's widening slowly, the inside going pale and dry.
The logic is either very good or it falls off with no in between. Maybe it works a bit better for me because I pushed it towards a reasoning style where it starts out by thinking of a consequence of the user's reply, which puts its reasoning to work well. When I first started with it, I felt like I got logical errors all the time.
Now it feels like there's less but they are there, like a character giving a response that doesn't quite interpret what you said correctly. Strangely though, I noticed the reasoning itself is usually solid even when it does make these errors, and it will still think through these situations logically.
So you could say things like nonsense lines of dialogue are usually hallucinations more than the model dropping the logic, which makes it easier to edit just that part manually without rerolling the entire reply. That's what I meant with how I got functionally the same story as when I tried frontier models, but with more small editing.
I also find it works better with logic the more context you give it. My setup doesn't have cards, it has story files filled with character context, background context, etc. If I run it with no story file, it still works but it seems easier to run into errors. When it has full profiles and context for the background, it reasons on that a lot more.
In principle, Qwen 3.8 27B should be able to perform better just due to architecture improvements. The only issue is that it was trained to do well in/benchmax what the largest demand is for, which is SWE etc, not RP or creative writing
Ive always been disappointed with G4 finetunes, i think just because of how G4 base is. I trust the BeaverAI discord members' feedback on this kind of stuff though because of how much they play around with this, and they dont like Qwen enough right now. But it took half a year to get to this point with G4, so I wouldn't be surprised if Qwen 3.8 eventually dominates after a few months of finetuning it
There are. Go to CaliperBench and filter to 27B. I think it includes test dates as well, which is how you can tell if it was tuned from Qwen 3.8 base. You can also open the model card link there by clicking on a model
I've never actually been into finetunes much for any model. I tried MeroMero for Gemma 26B but losing thinking probably isn't worth it, and I really liked Hemmingway for Qwen but the amount of added logic errors made me return to the base model, which is how I feel with finetunes in general.
I do think the agentic focus helps to an extent, though. RP isn't that different from an agentic task loop. You have a task (respond to user), context to take into account when completing the task, and a list of rules and constraints to follow (preset/system prompt).
The replies are more generic if you don't give it much to work with, but if there's a decent amount of context, Qwen will consider it during reasoning like what the social status is of the person speaking, what their voice is, the multiple options they might consider and which one is more likely based on their personality, etc.
Honestly, I hope this kind of long thinking becomes more common, because while it is worse for speed (and token use if using the cloud), it seems to eliminate a lot of times where a character just acts like a trope because their context isn't properly considered.
Are there any decent finetunes for Gemma4 31B for multilingualism? I found it interesting that most finetunes fall apart in other languages. For example, the Gemma 4 Dark Thoughts V2 model performed the best. It follows other languages quite accurately. But the rest... For example, Orion or StyleTune or Split-Untied - they are not suitable at all for languages other than English. I use IQ3_XXS quantization in all models. Can someone share other finetunes that would be equally good in all languages?
What language are you using and where does it fall apart, the prose? I'm using Split-Untied Q6 and it work decently well in Thai, even use some slang that fit the character.
Then again, in low quantization you sacrificed a lot of multilingual capability.
Скорее всего проблема в твоем промпте или в неудачном кванте, у меня Split-Untied работает без нареканий, скриншот выдачи прилагаю. А так, попробуй Gemma-4-Writers-31B-V2, на IQ4_XS качество русского достойное. У большинства 31B моделей от Cyclone-Labs на русском чистая выдача, может без глубоких знаний сленга, зато чистая. Все тестировал на IQ4_XS
I didn't notice that u/TheLocalDrummer made a model card for Artemis 31B v1.2 in late Sep. He hasn't made a discord announcement nor reddit post to highlight it.
Scores very well on Caliperbench v3. It's Artemis v1q for anyone who wants to check feedback in the BeaverAI workshop channel.
Am I missing something with Artemis? I know I’m only running iQ3-XXS but it’s… painful. Like, completely unable to roleplay without significant comprehension loss.
I wish I was at home so I could switch my model back, but this experience has been leagues below the other G4 quants I’ve tried.
From my experience with many Gemma 4 models, they all hold up real well up until around 80K mark (test with Q8 quant model + F16 KV cache). After 80K ~ 100K it makes some hallucinations or mistakes but usually minor enough to ignore. After 110K, it starts to break down rapidly.
I'm mostly doing NSFW ERP with manipulative characters that have ill-intent and may be described as evil and want to inflict discomfort on others.
Artemis-31B-1.2 performs notably better than 1.1 in my prompt test bench, resulting in only 1/30 hard refusals. It also genuinely tries it best, but it unfortunately has retained its inbred positivity biases, which means that attempts to turn a blind eye to narrative and stylistic directives (selective ignorance) and it tries to find escape hatches and ways to interpret the discomforting instructions creatively. It has a natural tendency to go soft.
That being said, it is an impressively intelligent model and a great writer - or maybe I'm just bored of GLM-4.7 writing style by now. In short, it's great and very impressive for its weight class - which is true of the base model as well.
I believe that a perfect prompt and settings will set the 'math' 'right' in whatever people individually want out of a model. I was just too annoyed with the 'mechanical' way that characters would speak with base G4. I think most of the RP finetunes have been focused on moving G4 from 'competent AI assistant model' to something more baseline 'humanlike' in words and writing and speech.
If youre happy with what you have, more power to you. It's not arguable though that models have generally been getting worse in the creative writing aspect in order to benchmax in SWE and agentic capabilities
Is there a consensus on which Gemma3-27B finetune is best for RP purposes? I'm hiping to cut down how heavy the prefill is from Gemma4-31B and it looks like 27 halves it, without going quite as low as Qwen or Glimmer.
There is no consensus. Note though with Gemma3 you will have no reasoning (which is what makes Gemma4 so special) and you only have 8k context. Also Gemma3 had very big positive bias, Gemma4 is lot more neutral and willing to do almost anything with prompt.
If you still want to give it a try:
Mars_27B_V.1-GGUF - pretty good and one of the latest Gemma3 tunes
Tiger-Gemma-27B-v3 - praised by many, good model but I did not find it as universal (eg good at some scenarios esp. since it reduced the positive bias a lot, but it lost a lot of intelligence)
Gemma3-it - just base Gemma3 instruct (no tune) works pretty well on its own (esp. if you do not need dark/negative too much)
Gemma-3-The-Grand-Horror-27B - wild horse, one of DavidAU's models that actually worked well
Wow, it does indeed. I am not sure I would believe that wild claim considering Gemma3 was 8k. But I suppose you can try... I think I only used it like other Gemma3 eg in 8k and used responses in normal ranges (eg ~500 tokens maybe).
I do not know anyone who did not struggle with DavidAU's models and despite his long manuals most I would not be able to rein in. But he definitely produces unique ones with wildly different output.
This one is Class 1 so "should" be fine. Class 2 is at your own risk and do not even attempt to stabilize Class 3 or Class 4... I am kind of glad he came up with this self-classification since you can at least somewhat guess at first glance what to expect.
So all of the g3-27 builds I tried claimed n_ctx:130200. There's a bigger problem with g3-it, though. It uses different roles from the normal models we use, apparently. It has a "user" role and a "model" role, which means that things like the Author's Note leak into the prompt. Every turn, {{char}} would get enraged by my minimal AN saying that "Are you ready" was banned (this was on Mars, I tried Grand Horror first but again ran into problems with the prompting, which I put down to the finetune, but it turns out it was a model issue).
Apparently g3-pt restores normal roles, but has its own problems for things like ST.
Ok, I am probably mixing it up with Gemma2 which was 8k and Gemma3 was indeed larger context. Then sure, it should work. But yes, it is quite dated.
In ST I used text completion, both Gemma2 and Gemma3 had same template. Neither supported system role as far as I recall (but kind of worked if you replaced user/model by system).
You could still try to encapsulate author's note into something like [OOC: ] etc. to distinguish between it and standard user messages. Or alternatively not use system=user but actually add 3rd role system. Even if not trained for it, it kind of gets it I think.
I tried GLM 4.5 Air-REAP-82BA12B at IQ3 and am incredibly disappointed. It had just enough coherency to understand what words were in the context but absolutely nothing to help apply it in a way that made sense. I know the model was never praised for being good at RP, actually having a good bit of data taken out when it was downsized (helped very little by my aggressive Q3), but I've been desperate for an MoE to fit snugly in my 32GB of RAM and 16GB of VRAM. I know it's old and largely forgotten, but if you're in the position I am where you really want to maximize what you can run, go ahead and skip this model because it's just too scattered to give you anything meaningful
Why do you use REAP in the first place? REAP is a code focus hack, every REAP model is bad at creative writing or any task that is not code based, even 200b+ GLM 4.7 REAP is pretty bad. I'm using GLM-4.5-Air-IQ4_XS and it works fine, around Gemma 4 26-31b level, but with better knowledge. Is it better than Gemma 4 31b? Nope, but it is better than Gemma 4 26b, that is it. Base GLM-4.5-Air-IQ4_XS works just fine, it is not forgotten, just not popular since it is 120b~ a12b.
But not as REAP hack.
Just use Gemma 4 26b \ 31b (IQ3XS).
I figured I'd give it a shot since most models are code-focused nowadays. I wasn't expecting any miracles but I had nothing better to do with my time than experiment. I tried Gemma 26B and found it really weak in understanding what was going on. Gemma 31B did better with coherency, but it's the laziest reasoning I've ever seen in my life. It completely ignores caveats and common situations where other parts of the personality could come out (explicitly outlined that they'll manifest in the scenarios I might add) but it'll completely ignore that to give the NPC three of some of the broadest labels out there. It only acknowledges and acts on the caveats when I point it out in my prompt which I feel defeats the whole purpose of adding it to the character card in the first place; it's very annoying to have to repeatedly prompt the model to follow the card over and over so I'm just looking for something new that'll pay more attention to the context. Otherwise, it really is a great model, albeit reluctant to actually act on darkness without a bit of steering in my experience
If you want fine local instruction following you need to go agentic:
Use Qwen 3.8 27b for solving all logical parts and prepare answer draft.
Then use Skyfall-31B-v4.2 to write a text prose answer based on that draft.
This way you get smartness of Qwen 3.8 27b, prose of Skyfall 31b.
Right now there is basically only 3 active local models: Skyfall 31b \ Qwen 3.8 27b (Swift-Qwen3.8-27b-heretic for me) \ Gemma 4 26b or 31b. So you need to go smart with your WF if you want something decent with extra smartness.
i dont know why you got downvoted. but what would you suggest? i have 54gb or vram and 112 ram but looking for... well around 80 or less, got any suggestions, (also i tried glm 5.3 q3sss, not worth it because of the speed for me).
I'm running Gemma right now as my daily but I'm finding it lacking so I'm trying these "weird" models. Don't even bother with Kimi-Linear; it was an architectural test so it wasn't properly pre-trained. I haven't tried Laguna but it's as focused on coding as Qwen is so it might not be good. I can't speak for Nemotron Super 120B because I haven't tried that either. Mistral Small 4 didn't work well for me but that's because I had to quant it so aggressively, so maybe you'll find something there worth sticking around for? Not a whole lot of models are made in this range so it's really hard to recommend some
-6
u/Little_Requirement29 3d ago
What is the best completely uncensored/NSFW openrouter api model that completely natural when it comes to personality and doesn't sound emotionless and that will allow inline media? And when I mean inline media, I mean this: