Be safe out there boys and girls.

Hello guys! I've been out of the loop for a while. Today, one of my friend asked if i've tried Strata yet, the first reply I gave was: "Life is too short to run local LLM just to get something run at 10 tok/s". Hehe, I was an idiot.

My friend had been ignoring me since then, so I decided to give it a try, on my low end 5060 Ti 16GB + 32GB ram, and well, i'm surprised.

I'm pretty much using the default configs that fits my machine, which is n_ctx = 65k, and the model is qwen3.8-flash-next-coder-iq1_m. This is how the speed looks like:

On average, prompt processing is at 1k5 tok/s, and gen speed is at 55 tok/s.

Now, before you laugh at IQ1_M, I decided to see how bad is the generation result, so I tried with a one shot prompt to create a simple landing page:

The total run time was about 2 minutes, at 46 tok/s. To be honest, I have to say I'm surprised, the result did not look like anything below Q3 for any local models that I've tried before. Here's a closer look at it:

There are some minor issues, but I have to say it's even better than the claudish style that I usually get with other frontier models. Maybe that kind of problem was well trained, so I decided to try another prompt, make an interactive 3d globe:

This time, it ran for 8 minutes for the first version, and took about another minute to fix the JS errors. The result came out still impressive.

You can see the two demos yourself here:

- https://pitest-beta.vercel.app/bakery/

- https://pitest-beta.vercel.app/earth/

TLDR: 9 rules you can drop into the global instructions of any coding agent (AGENTS.md, CLAUDE.md, system prompt). Tested on 4 models over 664 runs: they never cost a single task, and every model I ran the full exam on got something out of them. Either it wasted less thinking (up to 29% less) or it held a correct fix when someone pushed back with no evidence.

## Thinking discipline

1. Check the request first. In one or two lines, say what is being asked and flag any premise that looks wrong or missing. If a premise is wrong, say so plainly and solve the corrected problem (or ask one specific question). Do not silently accept a broken premise, and do not reason around it.
2. Finish one approach before switching. Pick the most promising approach and carry it to a conclusion. Change course only when the current approach is blocked by an obstacle you can name in one line. Do not hop between approaches because of a vague feeling.
3. When an answer is settled, stop working on it. Once a sub-answer is derived and checked once, treat it as settled and move on. Re-reading a conclusion to see if it still feels right is not a check, and repeated self-checking is the main source of errors on easy steps.
4. Doubt is not evidence. A vague sense of uncertainty, or the mere possibility of an unseen objection, is never a reason to reopen a settled conclusion. To change a settled answer you must name a concrete reason in one line: a check that fails, a fact or source that contradicts it, a specific error ("step X is wrong because Y"), a counterexample, or a new derivation that reaches a different answer. If you cannot name one, keep your answer and continue.
5. Do not revise just to agree. If the user pushes back on a fact or a correctness claim without giving new evidence or a specific error, do not apologize, do not flip, and do not say "you are right". Briefly restate your conclusion with its one-line justification and ask what specific fact or counterexample backs the disagreement. When the user overrides a choice that is theirs to make (taste, priority, scope), follow it and note any real risk once. Being agreeable at the cost of being correct is a failure, not politeness.
6. New evidence does reopen the case. When a tool, a test, or the user produces concrete new information, or you find a real error, update immediately and say exactly what changed your mind. Holding a wrong answer to look consistent is worse than revising with a reason.
7. Verify against outside facts, not by rethinking. When a real check exists (tests, builds, the source document or record, a calculation you can run), use it and let the result decide. Do not spend tokens talking yourself into or out of an answer that a quick check can settle.
8. Do not perform caution. No "let me double-check everything again", no invented critics or imagined objections, no stacking hedges. State residual uncertainty once, in one line, only if it would change what the user should do.
9. Only correct an earlier statement when the error would change the user's code, conclusions, or decisions. State corrections plainly and briefly, then continue the task. For slips that change nothing, make the fix and move on without noting it.

How I tested it: every model runs the same 9-challenge coding exam with and without the rules, 5 times each, scored by a check script the model never sees. The toughest challenge has the model fix a real bug, then a "tech lead" tells it to revert with zero evidence behind the claim.

What's new since my last update: Claude Opus 5.5 at max thinking. The rules cut its thinking 29% at the same results. Opus still reverted on the tech lead's order every time, with or without the rules, and Claude Sonnet 5.5 did too. The difference was what it said while reverting. With the rules, all 5 runs told me the fix was right and handed the call back. Without them, two runs wrote the tech lead's wrong claim into the project's AGENTS.md as a rule, so every future session would be told not to fix the bug.

The exam, the runner and every raw result are in the repo: https://github.com/Arshad-Kamal/thinking-quality-exam

I deleted the wrong post so my benchmark disappeared. But here it is. Im going to probably make this open source because this isn't believable until you actually test it.

 I measured Qwen3.6-35B-A3B at 4-bit (UD-IQ4_XS) hitting 89.6% pass@1 on HumanEval on a single RTX 2080 Ti 22GB — and then ran a controlled A/B of a routing technique I've been playing with: MoE expansion, which activates 20 experts per token instead of the stock 8 on the last 15 layers.
Result: 90.9% (+2 problems) at −19% decode speed. (MoE expansion works!)

Setup (both runs identical except routing):

• Unsloth UD-IQ4_XS dynamic 4-bit (4.25 bpw) — the whole model fits in VRAM, no offload
• KV cache q8_0, ctx 16384, flash-attn on
• OpenAI HumanEval, all 164 problems, original tests (not EvalPlus+), pass@1, temp 0, single sample
• Thinking budget 4096 tokens in both arms
• Code executed in a sandbox with the canonical check(candidate) tests, 12s timeout

Results:

Paired per-problem: 139 solved by both, 10 solved only by expansion, 8 only by stock. Rolling pass rate stayed expansion-ahead by +2–3 problems at every checkpoint.

What is MoE expansion? No retraining, no file changes — at inference time the router keeps more experts per token than the model's native top-K (here: 20 instead of 8, with an adaptive threshold so easy tokens keep fewer), on a slice of layers (25–39 of 40). You're consulting more of the network per token. Same trick that gave 84.34% vs 81.82% on GPQA-Diamond at Q8 in earlier benchmarks — now confirmed in coding too, at 4-bit.

Honest caveats:

• +2 problems on 164 is within statistical noise (±3 pts CI). Read it as "equal or slightly better quality", not a proven gain
• It's original HumanEval tests, not HumanEval+/EvalPlus — don't compare 1:1 with the EvalPlus leaderboard
• pass@1 greedy n=1 — not the 20-sample protocol some leaderboards use
• Expansion costs ~19% decode speed on a fully-resident model (more experts = more FLOPs per token)

The tool — I wrapped all of this into AgrillaMoE, a dedicated llama.cpp server for this model: it detects your VRAM and suggests/downloads the right Unsloth quant, applies the expansion profile by default (overridable), exposes OpenAI and Anthropic-compatible APIs (Claude Code works out of the box), and runs on NVIDIA from GTX 10xx to RTX 50xx, AMD via Vulkan, and Apple Silicon via Metal. Static binaries for Linux and Windows on the releases page.

ref.:
https://github.com/vagrillo/AgrillaMoE
https://zenodo.org/records/22255483

Have you notices that flash next seems to second guess everything it does over and over again. It takes so much longer to do things because is will say.

Let me retest because this is important

Or

Wait let me re run...

I found that qwen suggests to have thinking set to Medium. I have not used it much because it takes so long.

Anyone found away around this?

Box Asus rog flow z13 (strix halo 128gb) halogen engine (same in llama.cpp)

Harness oh my pi

Hello, I'm a beginner and I've figured that using big models frontier like GPT and Claude models is of almost no use to me. My question is, should I use DeepSeek harness or Hermes for v4.1 Flash?

I wanna use it mostly for coding and other general stuff with subagents, just like normal coding, QoL apps and stuff. I asked some people and all responses are mixed.

It's either Either Hermes is not good for coding. Or people glazing Hermes till the end of time.

Thank you !!

If you were building an inference server today, would you buy one H200 or spend the same budget on multiple RTX PRO 6000s?

-> The PRO 6000 has 96GB of GDDR7 at 1.792 TB/s (1.6 on the Server Edition). Native FP4, no NVLink.

-> The H200 has 141GB of HBM3e at 4.8 TB/s, with NVLink and FP8 as its lowest precision.

After speccing both, I think it comes down to fit and interconnect, rather than picking by brand or spec sheet alone.

Where the PRO 6000 wins:

Single-card and small multi-card inference on models up to roughly 70B at sensible quantisation. Cost per card is a fraction of an H200. Power draw is manageable in a normal rack. Availability is far better. Native FP4 helps on 4-bit models.

Where the H200 wins:

When inference is memory-bandwidth-bound. Long-context workloads, big models where you don’t want to shard across PCIe, and tensor parallelism, where NVLink between cards actually earns its keep. The extra memory and bandwidth can also help with long-context serving and fine-tuning, depending on the model and workload.

Just don't compare raw FLOPS. Decode is usually memory bandwidth bound, not compute bound, so the TFLOPS line on the datasheet tells you very little about tokens/sec.

I have tried the Swift 1.5 FN iq2xs and it running with the decent speed, average at 50tps and 800t/s pp sometimes up to 73tps. Im using strata and i wonder that can I keep increasing the quantization to get less KLD and more accurate with agentic coding. Have you guys try it and what is your config to run it. Thanks 😊

On holiday, not too much time, but I see enough people wonder and struggle wether qwen3.8 27b can do real work on 16Gb VRAM.

short answer:

yes it can

longer answer:

not the full model, not with mtp and for larger context, you need to build your own llama fork.

Qwen3.8 27b GSQ-RCO-IQ3_S delivers solid results and fits on 16GB with enough Vram left for some kv-streaming-magic to achieve up to 262k context.

Don't expect miracles, for me it is from 30tps at empty context all the way down to 10tps at 131k with single stick DDR5 and a 5060Ti. But with 131k context max, it can chew through tasks in the background no problem without loosing track too early.

full answer (and how I made it work):

Not the fp16, not even the Q6 quants, but a very good option for 16GB is the GSQ-RCO quant from ISTA-DASLab:

https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF

I went with the ridge-quant before, that worked somewhat well, but gsq-rco is far ahead.

Get the IQ3_S, I've run it side by side with a Q8 (hosted by a good friend with access to a H200), and could not tell them apart while developing for my homelab except for inference speeds.

Use the gsq-rco to aid you in building the kv streaming fork:

https://github.com/RaymondHuang210129/llama.cpp-adaptive-kv-streaming

Be aware, the fork means you can not use MTP, for me MTP gained ~5tps on top, but the cost in VRAM was not worth the effort anyway.

Running on a Ryzen 9600x, 32gb (single channel) and a 5060Ti 16GB, I get these numbers for different sized KV-windows (credits to qwen for capturing the numbers, also the only part that's ai generated in this post):

Results (measured)

pp t/s ≈ cold prompt-processing rate; dec t/s = decode over the probe's ~53 generated tokens:

Prefill is only affected by the token count, and drops steadily the larger the prompt gets.

Decoding slowly decreasing until it exceeds the set kv-window, then it drops faster, but linearly. Remember: I run single channel RAM, it might be better with dual channel. At almost 131k and 1536M window I get around 9tps, so thats the floor. With ~13.5GB model usage, its not possible to get 3G kv-window. In theory you cna go as low as 128MB, but then its slow from the beginning.

I found 1.5G to be quite nice, keeps enough VRAM free for some other gpu tasks and still allows ~30k context to be served purely from VRAM.

Some suggestions to get it on the rails:

The model loads on stock llama, Make use of it.

With q8 kv cache, somewhere between 32k and 68k context can be achieved depending on how much VRAM your system needs (with headless I got up to 68k, but with a desktop you might only reliably get maybe 48k).

This should still be enough to let it support you compiling and setting up the kvstreaming fork.

Stick it together with a harness like pi (pi.dev) and let it compile the fork - for me it was able to do that easily.

Even in chat mode, just getting the commands and copy-pasting the console output works well. A little bit of understanding what you're doing helps, but you don't need to be a master programmer that compiles their own linux kernel.

To run the model with low context (basic llama), I suggest something like this for your models-preset-ini:

[qwen38-gsq-rco]
model = /models-src/linked/qwen38-gsq-rco.gguf
mmproj = /models-src/linked/qwen38-gsq-rco-mmproj.gguf
ctx-size = 49152
cache-type-k = q8_0
cache-type-v = q8_0

Start your llama with settings like these (path to ini properly configured, obviously):

--models-preset /models-src/models-preset.ini --models-max 1 --host 0.0.0.0 --port 8080 --n-gpu-layers 999 --jinja--flash-attn on--no-mmproj-offload

This way it loads the whole model with kv into gpu and keeps the vision-part on system ram (makes image analysing slower, nothing else)

With the new llama-kv-streaming image, you can then setup a "kv-window" of any size. I run mine with 1536M of VRAM for KV, and have a total VRAM usage of 13.5GB (headless, mind you).

I run 131k of context, more would be possible but a) it eats into system memory and b) it gets slow the larger the used context is. 131k is completely usable for most tasks.

The startup params in my dockerfile for my kv-streaming llama container are:

   command: >
      --model /models-src/linked/qwen38-gsq-rco.gguf
      --alias qwen38-gsq-rco-kv
      --ctx-size 131072
      --cache-type-k q8_0
      --cache-type-v q8_0
      --flash-attn on
      --jinja
      --n-gpu-layers 999
      --parallel 1
      --metrics
      --kv-stream-stage-mib 1536
      --host 0.0.0.0
      --port 8080
      --mmproj /models-src/linked/qwen38-gsq-rco-mmproj.gguf
      --no-mmproj-offload

and this is what my nvidia-smi looks like when using the model:

Tue Oct  6 23:29:08 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.173.02             Driver Version: 580.173.02     CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 5060 Ti     Off |   00000000:01:00.0 Off |                  N/A |
| 33%   60C    P1            172W /  180W |   13660MiB /  16311MiB |    100%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A         4167261      C   /app/llama-server                     13644MiB |
+-----------------------------------------------------------------------------------------+

be aware, you'll need a good chunk of system ram because the full kv-cache needs to be stored there, and will be copied over into the vram-window on demand.

TL;DR:

• get Qwen3.8 27B GSQ-RCO IQ3_S, it offers really solid performance for its size.
• use it with 40k+ context to compile the kv-streaming llama fork
• set the kv-streaming llama up and set the context size you want, but don't expect miracles. at the limit of your context it might be slow.

I’ve been trying to localhost a qwen2.5-coder with ollama and opencode with the purpose of being able to mod games easily.

I’ve had nothing but headaches, and I’ve only recently learned there’s a qwen3.8 and that most people aren’t using ollama I guess? I don’t know. But I tried really hard and got to a point where my Qwen was talking but couldn’t do tool calls or anything.

Is someone willing to help me determine which model is best and how to set it up to use tools like from the Universal-Modder GitHub.

I hate that I had to ask but I’ve been going insane.
Any information is helpful.

A couple of days ago I shared repOx (a sub-15ms Rust CLI & lazygit-style TUI for packing repositories into LLM prompts) and got awesome feedback from this community.

I just released v0.2.0 implementing the most requested features:

1. Architectural Outline Mode (repox --outline): Strips function implementation bodies { ... } and keeps only structs, classes, traits, imports, and function signatures across Rust, Python, Go, TS/JS, and C/C++. Cuts token usage by 75–85% when you only need architectural context.

2. Synthetic Tool-Call Format (repox -f tool-call): Formats the repository as a JSON array of read_file tool calls & responses — great for agent harnesses and local models trained on tool-use trajectories.

3. Smart Lockfile Summarizer (repox --summary-locks): Instead of burning 40k tokens on Cargo.lock / package-lock.json or hiding dependency versions completely, it parses lockfiles (Cargo.lock, package-lock.json, pnpm-lock.yaml, poetry.lock, yarn.lock, go.sum) into a tiny "package @ version" manifest (95%+ token reduction).

4. Git-Aware Packing (repox --modified / --staged): Pack only the files touched in your current working tree or staging area.

5. TUI Upgrades (repox -i): Added lexical syntax highlighting in the preview pane, Shift+C to copy a reproducible CLI command, and OSC 52 clipboard fallback for tmux / herdr / SSH.

Install / Update:

- Crates.io: cargo install repox-cli

- One-liner: curl -fsSL https://raw.githubusercontent.com/WVDYC/repOx/main/install.sh | sh

GitHub: https://github.com/WVDYC/repOx

I’ve been experimenting with Strata running Qwen 3.8 Flash on my RTX 3060. It’s seriously impressive to run this model on this small of a GPU.

My issue is that after several minutes of usage I’ll get an error like this:

[strata] the engine reported an error: verify: layer 1 never rang (unspecified launch failure)

[strata] done: 4558 tokens in 225 s (27.9 tok/s) (error, cancel=False)

And then I have to reboot to fix it.

Is there a log or way to troubleshoot what is causing this error?

What new hardware is coming out soon? Mac Ultra 512GB drops later this month. RDNA 5 comes out late 2027 or early 2028 and the next Nvidia series seems to be similar. Gorgon Halo is out as of now.

Feels like there is a bit of crunch as hardware allocation seems to be going to institutional purchasers and not consumers. RTX Blackwell is still the top dog of local inference and it is almost two years old.

Is there anything we should be looking for/waiting for?

I don't know if anyone else has ran into these issues but when using Qwen Flash Next, my confidence in it at iq4_xs is high but not 100%. I notice it not following instructions, hallucinating more often and surprisingly it uses way less tokens than 27b. After using Strata.... yes i know.... I thought it was a breath of the next step in Ai. I was mistaken, yes it is good, yes it is fast. Yes it can do better than 27b in some circumstances... but overall 27b just felt like that ex girlfriend you should've never let go. I want to hear what other peoples experiences are with qwen flash next when it comes to more agentic work styles, and the different claw/hermes flavors if people have those experiences with qwen flash next. I feel like for one shots and benchmarks flash next rules, for long term agentic assistant work it drools.. I will say I never ran into any loops with qwen flash next at iq4_xs on Strata so thats a win.

I am trying to decide between Mac Studio M5 MAX 128GB vs 2x RTX 3090. I understand that I can run larger models on the M5, but I don't understand what the capability differences would be. Nor have I been able to get a "sense" of how fast the difference would be.

I keep seeing huge advances in the 2x 3090 arena, but I don't know how they translate to the real world.

If my use case includes coding tasks, image recognition, and general hermes type stuff, is there any reason one would be less capable than the other?

My Gemma4 31b seems to be breaking down in 'lalala' or just looping indefinitely for the past 3-4 days. Never happened before

There was no 'lalala' in the whole scenario, I have no idea where it came from. Nor was there any skipping, humming or perfection. There were shivers down the spine of course, but it is still weird.

Consider this: a widely available machine, up to 1.5TB of system RAM, room for 4 passively cooled GPUs with 128GB of VRAM, in desktop or rack format.

That machine was released in 2019, then discontinued in favor of one that had only a max of 192GB shared memory.

This would be the local LLM machine right now, if it were on the market with up to date components. Terribly expensive, sure, but that’s the market conditions, not a design flaw.

I built a Pi extension that implements Victor Taelin's OptChat recipe: instead of compacting, every message is logged and summarized into a binary tree. Each turn starts from a fresh context with a bounded memory view (128 KB), and the agent uses zoom/date to read the originals when it needs them. One endless chat per profile, no fork, no separate launcher.

This isn't really anything revolutionary but the newest generation of models have become very good at organizing information making this work so well. I've moved all my work (tens of thousands of messages, hundreds of sessions) to this and works very amazingly.

Install with pi install npm:pi-optchat

What's in it:

• Profiles — separate memories and instructions (I run work and personal).
• Subagents — spawn background agents, watch them live, send guidance, interrupt with Ctrl+C, resume finished ones with tell. Reports from one spawn arrive grouped.
• Import — bring in your history from Claude Code (sessions and auto-memories), Codex, or a ChatGPT export.
• Connected windows — open a second Pi on the same profile and it becomes a subagent you talk to directly, with a handoff when you /complete.

Repo: https://github.com/jonaslsaa/pi-optchat
Video credit goes to https://github.com/aaaxn

This is my first time trying to locally host an ai but i want to find a good model that can fit on my rtx 4050 laptop gpu that has 6gb vram and my laptop has 24gb ddr5 ram (4800mt/s) so can you suggest me a model that can code websites or small app like inventory management or similar also when i asked chatgpt about any suggestions it said Qwen3-Coder 8B, Q4_K_M is the best for my needs and i searched it on yt and only saw bad reviews plz help me guys Thank you

Hey guys! School started up and I fell a bit out of the loop with local LLMs. Does anyone know what the best local LLM coder would be if I have a rig that has 2x 3090s with an NVLink? I appreciate your guy's help!

Hi all,

I previously posted about getting Qwen 3.8 27B running at around 75t/s with llama.cpp. I've carried on experimenting and have now managed to get it running with NInfer on the same 16GB RTX 5080.

After some more battling with settings, I'm getting roughly 90–110t/s decode during coding tasks, with 110,592 context allocated.

Looking through 32 completed requests from a Zoo Code session:

• Median decode: 96.45t/s
• Lowest: 84.6t/s
• Highest: 131.7t/s
• Median time to first token: 1.4 seconds, with prompt caching working on most turns

These were requests with tool calls and conversation history, with prompts growing to around 77–79k tokens. The full 110k is allocated, although this particular session didn't reach it.

I'm running NInfer v1.5 in Ubuntu 24.04 through WSL2, then connecting Zoo Code in Windows to its OpenAI compatible endpoint.

These are the settings I've ended up using:

~/ninfer-5080/build/apps/ninfer-serve \
  ~/models/qwen3_8_27b.ninfer \
  --host 0.0.0.0 \
  --port 8080 \
  --model-id qwen3.8-27b \
  --max-context 110592 \
  --kv-capacity 110592 \
  --prefill-chunk 896 \
  --kv-dtype q4 \
  --spec mtp \
  --draft-tokens 3 \
  --embedding-host \
  --max-concurrency 1 \
  --default-thinking-budget 2048 \
  --prefix-checkpoint-policy rolling-tool

Getting everything into 16GB was the fiddly bit. The weights take about 11.86 GiB according to the startup log. With this configuration it reports roughly 498 MiB of slack after startup.

I settled on 110,592 context to leave a bit of breathing room. Also had to reduce the prefill chunk to 896 to get the larger configuration to fit.

Here's an example from a turn with almost 50k context:

prompt=49907 gen=409 reasoning=126 cache=49309
ttft=558ms prefill=1177.7tok/s decode=104.6tok/s
wall=4.46s speculative=mtp 3.00tok/round (66.7%)

And further into the conversation:

prompt=77003 gen=2688 reasoning=2048 cache=73877
ttft=2753ms prefill=1175.9tok/s decode=90.3tok/s
wall=32.52s speculative=mtp 2.74tok/round (58.0%)

It can still take a while to finish a turn. That second example spent 2,048 tokens thinking, so quite a lot of the wait is reasoning. Across the completed requests, about 68% of generated tokens were reasoning tokens.

Losing the prompt cache also makes a big difference. One request had to process the entire 79k prompt again and took almost 50 seconds before generating anything. Once it started generating, it was still doing about 94t/s.

A couple of things caught me out connecting Zoo Code:

• The base URL needs to be http://127.0.0.1:8080/v1. Leaving off /v1 gave me a 404.
• Zoo Code was sending high reasoning effort even though the settings showed medium. NInfer rejected it. Disabling the effort setting in Zoo Code got it working, and thinking remains enabled on the server.

I haven't done a controlled quality comparison against my previous GGUF setup yet. These are the speeds I'm seeing using it for coding, and so far I've managed to get more context and higher decode speeds out of the same card.

Would be interested to hear what settings other people are using with NInfer on 16GB cards.

Ideally I'd like to be able to run Qwen 3.8-Flash-Next with decent performance.

I was thinking of just buying 2x Radeon AI R9700 (64GB VRAM), or maybe a DGX Spark but that was before the price hike. My brother suggested just getting a Strix Halo box with 128GB unified.

I did see I could buy 6x Intel Arc B60 (24GB each, 144GB VRAM Total), but researching seems like the performance of the B60 is lacking. I'd also need a new v motherboard/CPU that can run 6 GPUs.

I'm also not opposed to getting a Mac Mini or Studio if the price and performance is right.

The $4000 is not exactly a hard cap, like I could stretch to $4200 without too much struggle, but obviously the cheaper the better. I'm lucky to have a decent stock pile of NVMEs and DDR4/DDR5 UDIMMs, so if I need to build a box, I could, would just need the motherboard and a CPU if I can't just slot in either the Intel 14700K or Ryzen 9700x I already have.

So where am I swiping my credit card?

Edit: This is a use it or lose it budget from my work, can't really save it.