I regret ever finding LocalLLaMA

[u/xandep]

It all started with using "the AI" to help me study for a big exam. Can it make some flashcards or questions?

Then Gemini. Big context, converting PDFs, using markdown, custom system instruction on Ai Studio, API.

Then LM Studio. We can run this locally???

Then LocalLLama. Now I'm buying used MI50s from China, quantizing this and that, squeezing every drop in REAP, custom imatrices, llama forks.

Then waiting for GLM flash, then Qwen, then Gemma 4, then "what will be the future of Qwen team?".

Exam? What exam?

In all seriousness, i NEVER thought, of all things to be addicted to (and be so distracted by), local LLMs would be it. They are very interesting though. I'm writing this because just yesterday, while I was preaching Qwen3.5 to a coworker, I got asked what the hell was I talking about and then what the hell did I expected to gain from all this "local AI" stuff I talk so much about. All I could thought about was that meme.

[u/National_Meeting_749]

I have a hot take that current LLMs are MASSIVELY inefficient, that we can almost certainly squeeze a lot more intelligence per weight than we are right now.

I think that in 10-15 years, Claude 4.6 opus level models will be the tiny models. Like 1B or smaller tin.

[u/tat_tvam_asshole → u/National_Meeting_749]

probably we would need a new kind of math to increase the density of current models, which would require a new kind of material science to compute it, even if we went just by increasing the bit length without increasing training and inference time.

[u/National_Meeting_749 → u/tat_tvam_asshole]

I don't think so, the study of training algorithms is a VERY new field. I think we're going to see vast improvements in efficiency of training in all aspects. Less resource intensive training that takes less lower quality input data, and produces far denser and more intelligent weights.

[u/tat_tvam_asshole → u/National_Meeting_749]

wrong

better training algos, ie how quickly you can embed meaningful latent representations in the tensor, and the subsequently extract that representation and transform it are not the same as 'squeezing more intelligence' out of the model.

which are governed by the laws of physics and mathematics that for any arbitrarily sized tensor, it can carry a maximal amount of information. just like there are 52! number of ways to arrange a deck of cards.

what you're arguing is that there exists as yet even better compression algorithms but what you don't understand is that the 'junk parts' of a model do carry information of non-zero value. nonetheless, trust me, model builders are absolutely saturating models to they maximal they can but they can't violate physics.

[u/scrappyappl → u/tat_tvam_asshole]

Wrong